Migrating from Apigee Edge to Apigee X
Like all technology, businesses must consider upgrading from old systems to new ones. This shift can range from a simple transition to a complex, multi-year project. By the same token moving your APIs from one API Management platform to another varies in complexity. API Management platforms have been in use since the late 2000s, with many businesses adopting them from 2010 onwards. Now, many of these companies are looking to upgrade to the latest versions of API Management platforms to take advantage of new features and benefits.
One example of this trend is the migration from Apigee Edge to Apigee X. Apigee Edge, a SaaS API Gateway, has been available since around 2014, and it was acquired by Google in 2016. In 2021, they launched Apigee X, fully integrated into the Google Cloud Platform (GCP). Although initial uptake was slow, there is now growing interest from Apigee Edge users in migrating to Apigee X.
Is migrating easy or hard?
It's somewhere in the middle. Although migrating from Edge to X is easier than to other platforms, it's still a complex task that requires careful planning to ensure a smooth transition of platform environments and APIs. Additionally, for many businesses moving to X on GCP, this may also be their first time using GCP services, adding another layer of complexity to the migration process.
Assess and Design
Assessing your current APIs, the Edge platform, how it connects to backend services and tools (like CICD and development tools), the developer portal, and the API users is the first step in understanding the project size and what's needed for designing the Apigee X platform and your migration plan structure.
Starting with all your API proxies you’ll need to discover how everything is plumbed together;
map all the backend services used in API proxies,
understand the network connectivity from the back-end services to the API proxies,
front-end connectivity for API access ,
API analytics and usage patterns,
understand the use of policies and shared flow policies with API proxies,
access to Edge Orgs and environments by users and and tools (IAM),
CICD and pipeline tool access to Edge,
monitoring and alerting of Edge and API proxies,
monetization and billing integration,
developer portal integration,
and release management implementation and process including the separation of Edge Orgs and environments.
What are the differences between Apigee Edge and Apigee X?
Differences between Apigee Edge and Apigee X
For example, in Edge, the system automatically handled load balancing, hostname resolution, and API security for your front-end access to APIs (configured through the Edge UI). In X, you'll need to set up cloud load balancing, host your TLS certificates, and connect Cloud Armor to achieve the same level of access and security.
Once you have explored and evaluated all parts of your Edge API Platform, you can start creating your new X platform with all the necessary components, like tools for continuous integration and continuous deployment (CICD). The solution design for your Apigee X platform should include typical architectural areas like requirements, assumptions, decisions, network design, security, and operational management.
Migration Plan
Now that you understand what you currently have and where you want to end up, it’s time to figure out how you’ll move from the start to the end. This is your migration plan. By now, you should know which parts can be moved in stages and which need to be moved all at once. Your goal is to minimize the need for big cutover events, as this reduces the impact on API developers and users. For instance, your plan might involve a script to copy API keys for authentication so that both Edge and X use the same keys for the same API products, allowing for the cutover of those APIs not to impact the API user’s authentication. If your APIs are all linked to the same web address without the option to direct traffic by subdirectory, then changing the domain name system (DNS) will require a complete switch to the cloud load balancer. However, even in this case, you can configure the cloud load balancer to redirect traffic based on subdirectories to APIs still on the original system. In essence, the migration plan needs to explain how all the components you assessed will move from A to B.
GCP Foundation
You might have Google Cloud Platform (GCP) already, but it's crucial to establish a strong base for your Apigee X platform. Google offers guides and Terraform templates to help set up your GCP service according to best practices and your company's security standards. This ensures that it's prepared for API migration. At the very least, replicating your Edge Org and Environment layout in Apigee X as Projects and Environments (in Environment Groups) through a lift and shift approach is the simplest foundation to go ahead with.
Migration
Front-end Network
This is one of the main differences between Edge and X. To make northbound networking work like it does out-of-the-box on Edge, GCP components need to be set up outside of Apigee X. Usually, a Layer7 Application Load Balancer is used for external load balancing with a Cloud Armor service as the WAF. Also, private northbound connectivity can be made more secure compared to Apigee Edge, so businesses can route private API traffic without exposing it to the Internet. There are different ways to connect Apigee X APIs externally and internally, which can make the migration more complex, especially with new options for differentiating traffic.
You can read this guide from Google on network options for Apigee X: Apigee networking options
Back-end Network(s)
As with northbound connectivity to your APIs on Apigee X, there are a number of patterns for southbound connectivity to your back-end systems. Most commonly back-end access is either through VPC Peering or a Private Service Connect (PSC) as a secure way to allow connection between GCP projects. Then using Cloud VPN or Cloud Interconnect to connect securely to your on-premise systems.
You can read more on deployment patterns and southbound connectivity option into Apigee X here: Private Service Connect deployment patterns and here Southbound networking patterns.
Edge to X
You should create scripts to move components from Edge to X. This allows you to run the migration multiple times, making it faster, more predictable, and error-free. To help get you started on scripts we’ve developed a couple of migration scripts for Companies to AppGroups and Company Apps to AppGroup Apps;
Thankfully there is great backward compatibility from X to Edge for all the components you will need to migrate across with a rich set of APIs available, so the scripting approach is a good strategy for migrating Apigee data.
You will also need to get quite familiar with both the Edge API and the Apigee X API. I recommend getting a hold of existing Postman collections to help you in developing the scripts you will need to migrate. Here are some Postman collections you can download, and a couple we’ve developed ourselves;
Apigee components and the recommended migration tactic for each
API Proxies
There is good support in being able to export from Edge and import API proxies to X. The sackmesser tool provided by Apigee is available for download and will migrate your API Proxies from Edge to X. Another method is using CICD tools and pipelines to push your code from a repository, package it up and using the Maven plugin to build and deploy to Apigee X. Google’s apigee-deploy-maven-plugin works with building and deploying API proxies to Apigee X. The Maven plugin may be a more palatable approach for most businesses as it will fit in with release management practices and testing the CICD integration to Apigee X can occur as part of the migration effort.
Recommended migration method: sackmesser tool, or your pipelines with the Maven plugin.
Shared Flows
Like API Proxies there is good support for out-of-the-box migration (exporting and importing). Again you can leverage the sackmesser tool provided by Apigee to migrate these, or the Maven plugin from your code repo.
Recommended migration method: sackmesser tool, or your pipelines with the Maven plugin.
API Products
A script will need to be developed to migrate API Products from Edge to X. Migration is fairly straightforward as all features on Edge API Products are supported on X API Products, further the API Proxies and environments do not need to be migrated or set-up as yet prior to migrating API Products. API Products are one of the first things you should migrate as Apps using those API Products will only be successfully migrated if an API Product of the same name already exists in Apigee X.
Recommended migration method: custom script
Apps (API Keys)
Before you migrate Apps both the API Products and Developer records should already exist on Apigee X. When creating a new App on Apigee X the API keys (credentials) will be a random string of characters which cannot be customized upon initial creation. Your script will need to remove this initial set of credentails and replace the key and secret with the version on the Edge App. Lastly, the API Products that you will need to add into each App will need to have their status updated to reflect the status in the Edge version of the App.
Recommended migration method: custom script
KVM
As KVM values are encrypted you will not be able to retrieve these fully from Edge, and the values will need to be retrieve from elsewhere. Furthermore once the KVM value has been created it cannot be updated, to update an existing KVM with a new value it will need to be deleted and recreated.
Recommended migration method: custom script
Companies and Company Apps
If you have monetization you will probably be using Companies and Company Apps. NOTE: this shouldn’t be confused with Teams when using the integrated developer portal, as Teams are only re-using Developer records to save the team construct.
Apigee X has recently launched AppGroups, currently in preview, as a similar construct as Companies. A set of scripts will need to be developed to migrate Companies and Company Apps. We have developed a couple of sample scripts you may wish to use and expand on here; Sonrai Migration Scripts
Before you can migrate Company Apps, you’ll need to ensure API Products and Companies have been migrated first.
Recommended migration method: custom script (get started with Sonrai’s scripts)
Developers
One of the benefits from using the Apigee Kickstarter Drupal portal is that it comes with the ability to sync developers in both directions; from Apigee Edge/X to Drupal and from Drupal to Apigee Edge/X. This saves a significant amount of hassles in migrating developers.
However if you’re using the integrated portal, you will need to develop a script to migrate the developer records across to X, furthermore developer records need to be present before you can migrate and relate Apps to developers.
NOTE: If you’re using the integrated developer portal, there are Teams represented in the portal. These Teams are simply Developer records. They will be represented with an email address in this format <guid>@devteam.apigee.io (e.g. 61741abc-1234-2c4c-be56-278f09f5148a@devteam.apigee.io).
Recommended migration method: Drupal portal sync (or custom script)
Developer Portal
There are two main types of developer portal: integrated and Drupal-based. The integrated portal is pre-built on Apigee and configured through the Apigee user interface for both Edge and X versions. The Drupal portal is self-managed, but Google offers a Drupal module for Apigee that lets you use your own Drupal CMS and customize it with the Apigee module for connecting to Edge (or X) and offers the same Apigee features available in the integrated portal.
Integrated Portal
Unfortunately there are two major drawbacks if you are using the integrated developer portal when it comes to migrating from Edge to X;
Content, styling and custom-code cannot be migrated through scripts as there is no API available to manage the integrated portal. Therefore all portal content must be manually migrated from the Edge integrated portal to the X integrated portal.
Developer login passwords are not able to be migrated if using the integrated identity engine with the integrated portal. What this means is the developers will need to re-register themselves on the portal. Once registered with that same developer email, any apps that were migrated for that developer will now be visible to them. However, if you’ve configured your portal to use a SAML/SSO provider for storing the developer identities and credentials then this is not something you need to worry about (developers will be able to use their existing password when logging into the new X-based integrated portal).
Drupal Portal
The latest version of the Apigee Module allows you to connect either to an Apigee Edge Org or an Apigee X project (and only one at one time).
Drupal settings for Apigee Edge/X connectivity
Many businesses that use Drupal for their developer portal usually host with Pantheon or Acquia. These hosting services allow for multi development environments, which makes the migration process much easier as you can develop and test the migration of data and the developer portal connection to Apigee X in a lower environment, prior to pushing the code upstream to higher environments, and then cutting over the live site from Edge to X.
Using Drupal environments to develop, test and then promote your site from Edge to X
Additionally, the Apigee module allows you to easily move (sync) developers. It synchronizes in both directions, so you can move developers from your Edge connected Drupal portal to Apigee X. Since you're using the Drupal portal, you won't encounter the same problems with content migration and developers needing to sign up again (as is found with the integrated portal). This makes the Drupal developer portal a great choice for migration.
NOTE: there is a sync of Team members (developers who are members of a Company) available. We have not tried using this feature and have instead relied on the Company-to-AppGroup script for migrating “Teams” and their “Team members”.
CICD
Development and deployment of APIs is typically done leveraging CICD tools and Google’s apigee-deploy-maven-plugin. Typically API developers are only provided open access to the Apigee UI for development in the lower environments, with higher X environments restricted to pipeline deployments and operational staff. Your pipelines will need to be reconfigured to deploy your API Proxies and Shared Flows to Apigee X. Be aware that the Maven plugin for Edge is version 1x, this will need to be upgraded to version 2.x for Apigee X, see here for further details.
Support
After you have successfully migrated and cut-over to Apigee X, supporting the Apigee X environment will be quite different to how Edge was managed. Firstly, GCP’s logging and monitoring is best place for operations to keep an eye on the Apigee platform and the APIs. This provide detailed insights into what is happening on the platform and with APIs, more-so than the Edge platform provided.
Furthermore Operations should get familiar with the increased analytics and metrics capabilities in the Apigee UI, in particular the new feature of Advanced API Security which not only constantly monitors and protects APIs from security threats, but provides detailed security reports for better actionable insights for Operations.
Opportunities for API program improvement
When considering a migration from Edge to X, it opens the doors to improving the state of the business’s API Program and so before embarking on a lift-and-shift migration the following should be investigated and improvements considered as part of the migration:
API Platform - access to the platform, development environments available, how APIs are exposed internally and externally, monetization of APIs, consolidation of other API platforms as well as Apigee Edge
Developer Portal - currency of CMS and API documentation, release management and content management practices, developer experience and developer engagement.
Release Management - DevOps and CICD practices with API development, tools used for API development and release, reuse of code, policies and shared flows, ease of release management stages and automation, practices that ensure high-quality and consistent APIs, speed of API development from idea to production.
API Standards - a rich set of standards and patterns that not only follow best practices but fit the current developer experience.
API Team - people who own and curate all aspects of the API Program and owners for each API product.
API Strategy - senior stakeholder support for the API Program, funding for the platform, APIs and incremental improvements, linking the API strategy to the business strategy.
Operate model - incident and problem management process and tools, service reliability and availability, support teams competency, training and scalability.