• Please wait …

Streamlining REST API Integrations in SAP Datasphere with Task Chains and FastAPI Microservices on SAP BTP 

Introduction 

With the recent introduction of API Tasks in Task Chains, SAP Datasphere now supports direct orchestration of external REST services (see SAP documentation). This enhancement unlocks new integration possibilities, particularly for scenarios where no standard connectors are available. 

In our use case, we aim to extract data from an open REST API and load it into SAP Datasphere. To achieve this, we developed a custom wrapper API—built with FastAPI and hosted on SAP Business Technology Platform (BTP)—that handles both data extraction and the writing of that data into SAP HANA Cloud, which underlies SAP Datasphere. The entire data loading process is then triggered and managed via a REST API step within a Task Chain. 

At the core of this solution is a lightweight microservice that seamlessly integrates a third-party REST API into SAP Datasphere. The microservice is designed to be modular, secure, and easy to maintain, and enables a fully automated integration pipeline. 

Key Features of the Approach 

  • REST API extraction wrapped in a FastAPI-based microservice 
  • Basic HTTP Authentication for secure access 
  • Data retrieval from public or external REST APIs 
  • Automatic detection and mapping of SAP HANA-compatible data types 
  • Dynamic table creation in SAP HANA Cloud 
  • Batch data insertion with upload timestamps for traceability 
  • Deployment on SAP BTP (Cloud Foundry) 
  • Registration as an HTTP connection in SAP Datasphere 
  • Integration into a Task Chain for full orchestration of the data workflow 
End-to-end architecture for REST API integration with SAP Datasphere: This diagram illustrates how external API data flows through an API-to-HANA Bridge into SAP Datasphere, enabling HTTP-based ingestion and orchestration with HANA Cloud as the underlying database.

Requirement 

Data integration from 3rd-party REST APIs is still a challenge in SAP Datasphere, particularly when no standard connectors are available. SAP offers solutions like Open Connectors or SAP Cloud Integration via the Integration Suite. However, these options come with additional cost and infrastructure overhead. 

Our proposed solution is a lightweight, developer-friendly alternative that utilizes: 

  • A simple REST service hosted on SAP BTP 
  • Open SQL schema access to the underlying SAP HANA Cloud instance 
  • Integration into Task Chains via the new REST API task type 

This allows full orchestration and governance from within SAP Datasphere, without introducing unnecessary middleware layers. 

Solution 

FastAPI Solution on SAP BTP 

At the core of this architecture lies a microservice built with FastAPI. This Python-based service acts as a universal adapter to extract, transform (lightly), and load data into SAP HANA Cloud. The code can also be found in our GIT repository. 

Key Components 

1. Authentication: All secure endpoints require HTTP Basic Authentication, with credentials configurable via environment variables.
2. HANA Integration:
a. Using the hana_ml library, the service connects to the HANA Cloud database and automatically detects column types using heuristics (e.g., timestamp recognition, length-adjusted NVARCHAR).
b. Dynamically creates a table if it doesn’t exist.
c. Inserts records in batch mode, with an added UPLOAD_TS timestamp column.
3. Endpoints:
Endpoint Method Description
/ GET Health check
/test-hana-connection GET Verifies database credentials and connection
/extract-and-write POST Triggers extraction from a third-party API and writes to HANA
/preview-data GET Fetches sample records from a REST API
/infer-types GET Infers HANA-compatible types from JSON response

4. Deployment: The service is packaged with a manifest.yaml for deployment on SAP BTP (Cloud Foundry). It uses the Python buildpack and runs via uvicorn in production.


 

Outlook 

This FastAPI-based solution can be flexibly extended to support more advanced integration needs: 

  • Authentication: Easily add OAuth2, API keys, or Bearer tokens to handle secured APIs. 
  • REST Call Chaining: Implement multi-step workflows by chaining dependent API calls asynchronously. 
  • Pagination: Add logic for offset, cursor, or token-based pagination to collect full datasets. 
  • Schema Flexibility: Support field renaming, normalization of nested JSON, and custom type overrides. 

Testing 

For Testing purposes we tested everything with Postman and the public available Harry Potter API https://hp-api.onrender.com. 

REST API response preview using JSON format: A Postman test fetches spell data from a public API. Each object includes an ID, name, and description—ready for processing and ingestion into SAP HANA.

SAP Datasphere Integration 

To communicate with this BTP service we performed the following steps: 

  1. Whitelist the IPs of our BTP instance in the SAP Datasphere tenant (For testing from Business Application Studio (BAS)you may also want to whitelist those IPs as well) 
  1. Create a HTTP connection for the usage in the Task Chain

Creating a generic HTTP connection in SAP Datasphere: This screenshot shows the setup for securely connecting to an external API using HTTPS, basic authentication, and credentials, enabling downstream API task automation.

In the task chain we can now add this step and configure the body. 

Integrating API access into a Task Chain: By selecting the "API Task" button, users can embed external HTTP API calls directly into their SAP Datasphere processing workflows.

In the Configuration panel we can now select our HTTP connection: 

Binding the HTTP connection to an API Task: This UI screen shows the selection of the configured HTTP connection (e.g., API2HanaCloud) that enables secure REST API access within the task.

And also configure our REST call: 

Setting up the API invocation details: The API Task is configured to POST a request to the API-to-HANA Bridge with parameters such as endpoint and target table. The synchronous mode allows real-time response handling.

After running the Task Chain we can see the data is now loaded into our table. 

Previewing ingested data in SAP Datasphere: The screenshot displays the contents of the HP_SPELLS3 table populated with API data, including spell names and descriptions, along with timestamps indicating when the data was uploaded.

Conclusion 

The integration of a FastAPI-based microservice into SAP Datasphere via Task Chains marks a significant advancement in streamlining data ingestion from third-party REST APIs. By leveraging SAP BTP for deployment and utilizing the new REST API task type within Task Chains, we demonstrated a lightweight and developer-friendly alternative to traditional integration tools like SAP Cloud Integration or Open Connectors. 

Looking ahead, the framework can be extended to accommodate more complex data workflows, robust security models, and advanced transformation logic. With this foundation, organizations can seamlessly orchestrate external data services within SAP Datasphere while maintaining full control, traceability, and governance—all without the need for additional middleware. 

Related Blogs

For Integration possibilities with Open Connectors and Fast API deployment you can also take a look at my other blogs 

Automating SAP Datasphere Task Chains with CLI Wrapping: FastAPI Integration with SAP Analytics Cloud 
Mastering Custom Integrations in SAP Datasphere: Leveraging Open Connectors for REST APIs 

Sources

SAP Community Links 

Author: Christian Schilcher-Willi

Christian has been working as a Business Intelligence Consultant at ZPARTNER since 2020. He is specialized in advanced SAP BW/4HANA, HANA native modeling and SAP Datasphere solutions. Christian has a strong technical background in ABAP programming, AMDP transformations, Python-based data processing. He has worked in projects in various industries and developed solutions for complex data extraction, integration and modeling.

×