Chargebee is a recurring billing and subscription management tool that helps subscription businesses streamline their Revenue Operations. It offers a variety of functionality, including:

• Managing recurring billing and subscriptions seamlessly
• Supporting hybrid business models
• Enabling expansion of global footprint
• Automating self-serve workflows for all use-cases.

Chargebee also offers native reporting functionality, but there may be times where one might have more complex reporting needs than what is available on the platform by default. Fortunately, Chargebee also has a marketplace of third party tools that you can leverage to meet a variety of needs, and they also offer an API that you can use to build your own!

This guide will walk you through one option for how you can build your own integration using Amazon Web Services (AWS). More specifically, you will be able to ingest data from your Chargebee account into AWS and use that data for your own custom reporting workflows.

Pre-Requisites

• A free AWS account
• A free Chargebee account
• Chargebee API Key
  • It is recommended to follow the best practices of least privileges when assigning access to your API key.
• Programming Fundamentals

It is highly recommended that the steps followed in this guide are done in your Test environment in Chargebee and a development account in AWS.

Architecture Overview

The architecture for this guide is very straightforward. AWS Lambda will initiate an export API call to the Chargebee API. A secondary Lambda will download those files once they are ready. All of your exported Chargebee files will be stored in Amazon S3. AWS Step Functions will orchestrate the entire workflow, and your Chargebee Site name and API key will be stored in (and referenced from) AWS Systems Manager Parameter Store. The parameters will be encrypted using AWS Key Management Service. An optional Amazon EventBridge Scheduler can trigger the workflow on a scheduled basis.

Implementation Instructions

Follow the step-by-step instructions below to create the integration. Click on each step number to expand the section.

• Step 0. [Optional] Import Sample Data to Chargebee

  You may already have sample data to work with in your Chargebee environment, and you are welcome to use that for this tutorial. If you do not have sample data, feel free to use the sample data provided in the chargebee-sample-customer-data.csv file. Follow the steps found in Chargebee’s Bulk Operations documentation to pre-load this data before continuing on.

• Step 1. Select a Region

  This application can be deployed in any AWS region that supports all the services used in this application (see the Architecture Overview section). We can refer to the region table to see which regions support these services. For the purpose of this guide, we will be creating resources in the US East (N. Virginia) region. We can select this region from the dropdown in the upper right corner of the AWS Management Console.

• Step 2. Create AWS Systems Manager Parameters

  AWS Systems Manager (SSM) Parameter Store provides the ability to securely store data such as passwords, database strings, and license codes as parameter values.

  In this step, we will use the AWS console to create SSM Parameters that will store the values of our Chargebee Site name and API key. We will later reference these values in our Lambda function code that will invoke the Chargebee API call.

  ---

  
  a. In the AWS Console, navigate to the AWS Systems Manager service. Click the Parameter Store link in the left hand menu. Then click Create parameter.

  
  

  b. On the Create parameter page, let’s provide a unique name for our parameter. For the purpose of this guide, we will use chargebee-apikey as the name. Keep the default Tier of Standard selected.

  c. Under Type, select the SecureString radio button. This will apply encryption to the value that is stored in the parameter. Under the KMS Key ID section, a default KMS key will auto-populate. This will be the key that is used to encrypt the parameter data.

  d. In the Value text box, we can enter the value of our Chargebee API key, then click the Create parameter button to create the parameter.

  
  

  e. Repeat steps A through E to create another parameter and store the value of the Chargebee Site name. By the end of these steps, we will have created two SSM Parameters as shown in the following image:

  

• Step 3. Create an S3 Bucket to Store the Export Files

  In this step, we’ll create a new S3 bucket that will be used to hold all of the downloaded Chargebee files.

  ---

  
  a. In the AWS Management Console, navigate to the S3 service. Then click the Create bucket button on the right hand side of the screen.

  b. On the Create bucket page, provide a globally unique name for the bucket such as chargebee-to-aws-tutorial.

  If you get an error that your bucket name already exists, try adding additional numbers or characters until you find an unused name. Also, make sure the Region you’ve chosen to use for this workshop is selected in the dropdown.

  c. Choose Create in the lower left corner of this page, leaving the remaining default options.

  Our newly created bucket will be visible the S3 console upon successful creation.

• Step 4. Create an IAM Role for the Lambda Functions

  In this step, we will create an IAM Lambda Execution role for our Lambda functions. This role defines what other AWS services the function is allowed to interact with (see Lambda Execution Role).

  For the purposes of this workshop, we’ll need to create an IAM role that grants our Lambda functions the following permissions:

  a. write logs to Amazon CloudWatch Logs

  b. read parameter values from Systems Manager Parameter Store

  c. write files to an S3 bucket

  ---

  
  

  a. In the AWS Management Console, navigate to the IAM service. Click the Roles link in the left hand menu, then click the Create role button.

  b. In the Select trusted entity page, keep the default selection of AWS Service.

  c. Under the Use case section further down the page, select the radio button next to Lambda. Then click the Next button.

  
  

  d. On the Add permissions page, under the Permissions policies section, search for and select the following Policy names:

  
  

  AWSLambdaBasicExecutionRole

  AmazonSSMReadOnlyAccess

  
  

  Then click Next.

  e. On the Name, review, and create page, we can provide a descriptive name for our IAM role such as ChargebeeExportExecutionRole. Then scroll to the bottom of the page and click the Create role button.

  We will be redirected to the Roles page once the role has finished creating. We can see this new role’s details if we click the View role button at the top of the screen.

  
  

  f. Under the Permissions policies section, we can see the two policies that we added in the previous steps. These policies are AWS managed policies, which is a type of pre-made, standalone policy that is created and administered by AWS. Using AWS managed policies can be a lot faster than writing our own policies, but there may be times where it is better to write a custom policy.

  We will now create a custom policy for the S3 permissions that we require for this workflow. First, click the Add permissions dropdown, and select Create inline policy.

  g. Then select Choose a service. Type in S3 into the Find a service search box and select S3 when it appears.

  h. Our download Lambda function will require a minimum set of permissions to be able to write files to our S3 bucket. We can add these permissions in the Actions - Specify the actions allowed in S3 section. Search for and select the following permissions:

  ListBucket

  ListAllMyBuckets

  PutObject

  PutObjectTagging

  PutObjectVersionTagging

  PutObjectAcl

  
  

  
  

  i. Select the Resources section. Keeping the Specific option selected, click the Add ARN link in the bucket section.

  j. In the Add ARN(s) pop-up modal, type in the name of the S3 bucket we created earlier in this guide (e.g. chargebee-to-aws-tutorial). Then click Add.

  k. Under Resources next to the object section, select the check box next to Any. Then click Review policy.

  
  

  l. On the Review policy page, we can provide a descriptive name for the policy such as ChargebeeS3WriteAccess. Then click Create policy. We should now see the custom inline policy in the Permissions policies section of the IAM Role details.

  
  

• Step 6. Create a Lambda Layer for your Lambda Functions

  Lambda layers are a convenient way to package libraries and other dependencies that you can use with your Lambda functions. For this particular integration, we will need the Chargebee API library as well as the Requests library.

  In this step, we will upload a pre-made Lambda Layer that includes both the Chargebee and Requests libraries. Creating one from scratch is outside of the scope of this tutorial, but you can find more details about this process in the Creating layer content section of the official AWS documentation.

  ---

  
  a. Download the chargebee-requests-lambda-layer from the application repo. In the AWS Management Console, navigate to the Lambda service and select the Layers link in the left hand menu. Then click the Create layer button on the right hand side.

  

  
  

  b. On the Create layer page, provide a name for your Lambda Layer such as chargebee-requests-layer. With Upload a .zip file selected, click the Upload button and select the zip file downloaded in the previous step.

  

  
  

  c. In the Compatible architectures section, select the check box next to x86_64. In the Compatible runtimes section, select the latest supported Python runtime (Python 3.10 as of the time of this guide’s creation) from the dropdown.

  You can learn more about these configuration settings in the Creating a layer AWS documentation.

  Then click Create.

  

  
  

  You should see the following screen once the Lambda Layer has been successfully created.

  

• Step 7. Create Lambda Functions to initiate and download an Export from Chargebee

  In this step, we will be making use of the Chargebee API to build the core functionality of exporting of data from Chargebee and storing it in AWS. The Chargebee API supports a number of programming languages. You’ll want to make sure you select both the Product Catalog version that is relevant to the version of Chargebee you are using as well as your supported programming language of choice to make sure you are seeing the correct documentation for your environment.

  

  
  

  For the purpose of this tutorial, we will be using the Product Catalog 2.0 version and the Python language of the documentation, and we will be specifically making use of the Export Customers API.

  We will be creating a total of two lambda functions, one that will initiate the export from Chargebee, and one that will download the data once the export is ready for download.

  ---

  
  a. In the AWS Management Console, navigate to the Lambda service. Then click the Create function button on the top right side of the screen.

  

  
  

  b. On the Create function page, provide a unique name for the first “export” Lambda function such as chargebee-export-function. Under Runtime, select the latest supported Python runtime (make sure it matches the same runtime as the one that was selected during the Lambda Layer creation process). Leave the default value selected under Architecture.

  

  
  

  c. Click the Change default execution role and select Use an existing role. In the Existing Role dropwdown, select the IAM role you created in the previous step. Then click the Create function button at the bottom right hand side of the page.

  

  
  

  d. In the lambda function console, scroll all the way to the bottom of the page until you see the Layers section. Click Add a layer.

  

  
  

  e. In the Add layer page in the Choose a layer section, select the Custom layers option under Layer source. Then select the lambda layer created in the previous section under the Custom layers dropdown. Select the latest version of the lambda layer in the Version dropdown. Then click Add.

  

  
  

  f. In the Code Source section of the lambda function, copy and paste the code in this file into lambda_function.py. Then click Deploy.

  

  
  

  g. Repeat steps A through F in this section for the second “download” lambda function. (Make sure to provide a unique name for it such as chargebee-download-function). Use the code provided in this file for the contents of this lambda function.

  

  
  

• Step 9. Create a State Machine to Orchestrate the Chargebee Export Workflow

  Now that we have all of our core resources created, this next step will bring the entire workflow together in an organized and procedural way. We will do this by leveraging AWS Step Functions, a serverless visual orchestration service.

  In this step, we will create a State Machine that will organize our workflow in a series of Steps that we define. The State Machine accepts a JSON object as input, and we will be using the following input object for ours:

  {
    "Status": "", # The current status of the export workflow
    "ExportId": "", # The Export ID that Chargebee will provide
    "Url": "" # The download URL that Chargebee will provide
  }
  

  ---

  
  a. In the AWS Management Console, navigate to the Step Functions service and click the State machines link in the left hand menu. Then click Create state machine.

  

  
  

  b. In the Choose authoring method page, keep the default value of Design your workflow visually, and keep the default value of Standard under the Type section. Then click Next.

  Defining your orchestration workflow as code is outside of the scope of this guide, but you can refer to the Amazon States Language for more information on this topic.

  

  
  

  c. The next page is the Step Functions Workflow Studio.

  

  
  

  We will use this visual designer to create workflow as shown in the AWS Step Functions workflow part of architecture diagram at the start of this guide.

  The first step we will add to our workflow is the “Export” lambda function we created in a previous step. We will do this by dragging and dropping the AWS Lambda Invoke action from the left menu into the designer form space.

  
  

  d. Next, we will add a Wait step. The purpose of this step is to wait a short period of time so that Chargebee has time to prepare the files to be exported and generate the download URL. To do this, type in “Wait” in the search field on the left hand side, and drag and drop the action under the Lambda Invoke from the previous step.

  
  

  e. After the Wait step, we want to add our second “download” lambda which will run after the conditions of the Wait step are met. We will later add a Fail state in case something goes wrong during the Export step, a Success state for when the entire workflow completes without error, and a Choice state to help the workflow decide between the previous two.

  For now, just repeat the same steps to drag and drop another Lambda step and place it under the Wait step.

  
  

  Now that we have some of the core pieces of our workflow added, we need to add the workflow configurations.

  f. Let’s start with the first Lambda Invoke. Select this item in the designer and give it a more descriptive name such as Chargebee Exporter Lambda.

  Then under the Function Name section, we will search for the name of the corresponding Lambda function that we created earlier in this guide (e.g chargebee-export-function).

  Scroll further down until you see the Next state section, keeping all of the default values along the way. Validate that the value of this drowndown is Wait. This is telling the service that once this Lambda has finished running, send it to the Wait step.

  
  

  Repeat the same process for the second Lambda in the workflow, adding a descriptive name such as Chargebee Downloader Lambda and selecting the corresponding lambda function (e.g. chargebee-download-function). For this Lambda, leave the default value of Go to end in the Next state section.

  g. The next step in the workflow is the Wait step. If we click this step in the designer, we can see the configuration options for this state. Under Options, we are able to define our wait to happen at a fixed interval or on a specific date and time.

  For the purpose of this tutorial, we will leave the default Wait for a fixed interval selected. Under the Seconds section, we will keep the default value of Enter seconds in the dropdown, and change our wait time to 10 seconds.

  If you have a lot of data in your environment, you can set your wait time to a longer period to reduce the amount of repetitions this state will run.

  
  

  h. If we scroll down to the Next state section, we will see that it is configured by default to go to our second Lambda Invoke. We want the workflow to check the status of our Export first before deciding if it’s okay to send to this Lambda function.

  To do this, we will add a Choice step to the workflow. Search for it in the search box and drag and drop it underneath the Wait step. Then if you click the Wait state item again, you should see that the Next state section now goes to the Choice state we just added.

  
  

  i. Now let’s configure the Choice state. If we click on this item, we can see under the Choice Rules section that this state let’s us define if-else logic to determine which state the workflow should transition to next.

  For our workflow, we want to transition to the Chargebee Downloader Lambda step if the value of Status in our input object is completed. We want to transition it back to the Chargebee Exporter Lambda if Status is in-progress. If there is any other value besides completed or in-progress in Status, then it most likely means there was an issue and we want to stop the State Machine with an error and skip running the Chargebee Downloader Lambda step.

  Let’s start by setting this error logic as our Default Rule. We first have to search for the Fail action in the search bar and drag and drop it into our workflow. You can drop it into the Rule #1 space for now as we will correct this in the Choice State configuration.

  
  

  j. Let’s go back and configure the Choice state. Scroll down to the Choice Rules section and click the edit icon on the right side of Rule #1. We are presented with the configuration options for this rule.

  Our Rule #1 will be to tell the workflow to go our download lambda when the status is completed. Click the Add Conditions button which will pop up a “Conditions” modal that we can configure to evaluate the value of our JSON input object.

  Keep the default value of Simple in the first dropdown, and validate that the value under the Not dropdown is empty. Under Variable, provide $.Status as the value.

  See Input and Output Processing in Step Functions for more information on why we are using this syntax to reference the value of our Status parameter.

  Under the Operator dropdown, select is equal to, and under the Value dropdown, select String constant. In the last box, type in completed. Then click Save conditions.

  Scroll down until you see the Then next state is section, and select Chargebee Downloader Lambda from the list.

  
  

  k. Scroll down and click the + Add new choice rule button. Repeat the same steps to add the in-progress condition, and set the Then next state is section to the Chargebee Exporter Lambda.

  
  

  l. To complete the Choice state configuration, set the Default rule to target the Fail step.

  
  

  m. For the last part of this section, we will add a Success state after the Chargebee Downloader Lambda that will stop the State Machine and mark the workflow as a success. Search for the Success action in the search bar, then drag and drop it under the Chargebee Downloader Lambda step in the designer.

  
  

  n. At this point, we now have our State Machine fully defined. If you click the Next button, you will be taken to a page where you can see your workflow design as code (similar to the below image). You can take some time to review the code definition before moving onto the next step.

  

  
  

  o. Scroll down and click the Next button again. This will take us to the final configuration settings for our State Machine.

  Let’s give our State Machine a descriptive name such as ChargebeeExportStateMachine.

  Under the Permissions section, keep the default value of Create new role selected. The Step Functions service will generate a new IAM role on our behalf based on how we have defined our State Machine. You will be able to see the name of the role that will be created as well as the permissions that will be added in the blue information callout at the bottom of this page.

  You have the ability to customize this role or create your own role should your State Machine definition change in the future. See the Creating an IAM role for your state machine documentation for more details.

  Under the Logging section, we can set our desired logging level. It is highly recommended to configure logging for your State Machine as it is important for things like monitoring performance and debugging. For the purpose of this tutorial, we will select the ALL option.

  Then click Create state machine.

• Step 10. Validate the End-to-End Workflow

  Now that we have created all of our integration components, we should be able to trigger the workflow and successfully export data from Chargebee. We will validate that everything is working as expected by executing our State Machine.

  ---

  a. Click the Start execution button. On the Start execution modal that pops up, you can provide a custom name for the execution run or leave the default auto-generated value. Under the Input section, copy and paste our input object into the text box:

  {
    "Status": "",
    "ExportId": "",
    "Url": ""
  }
  

  Then click Start execution.

  
  

  b. We will be directed to the Execution Details page which shows the details for all in-progress and past State Machines executions.

  If we scroll down to the Graph View section, we can see a graphical representation of the execution workflow that helps us keep track of the execution status. From this view, we can choose any step in the workflow to view details about its execution in the Step details component.

  
  

  Scrolling further down the page, under the Events section we can see a list of events associated with the execution history similar to the below image:

  

  
  

  c. To verify that the workflow was successful, navigate to the S3 service and click on the name of the S3 Bucket we created in an earlier step (e.g. chargebee-to-aws-tutorial).

  If you click on the exports folder, you should see a series of CSV files related to the export query we made earlier in the tutorial.

  
  

  d. We have validated that the Chargebee files were successfully exported and downloaded into S3. Now we need to check that our failure use case also works as expected. We will do this by running a new execution with an input object that will simulate an error from our Exporter Lambda. Copy and paste the following code into a new execution:

  {
    "Status": "error",
    "ExportId": "",
    "Url": ""
  }
  

  Earlier we configured our Choice state to trigger a failure in the State Machine if the value of the Status parameter was anything other than in-progress or completed. With the above input object, the State Machine should fail at the Choice state. We will see something like the below image:

  

  
  

• Step 11. [Optional] Add an Event Bridge Scheduler

  The content of this section is currently a work in progress.