CloverETL is now CloverDX - Learn Why

Back to CloverDX Blog on Data Integration

CloverETL and Facebook Graph API

Posted by Jan Sedláček on Mar 11, 2014 12:08:44 PM

Today we'll tackle how to utilize CloverETL with Facebook. Facebook Graph API allows applications to access social graphs. This blog post describes how to use Graph API within CloverETL.

Querying Facebook via Graph API consist of two steps:

  1. Generate an access token using App ID and App Secret
  2. Use the access token to invoke API calls

We’ll use App Access Token, as it doesn’t require user interaction during the generation process and does not expire. Check out the documentation to see the differences between available tokens and their limitations. For example App Access Token does not allow to use "me" as userId or search queries are limited to Page and Place type only.

Register the Facebook App

Let’s get to it. First, you need to register the Facebook app. Note that you have to be a verified Facebook user to be able to register the application. Go to and register a new application there.

CloverETL and Facebook Graph API

After registering your application, you’ll see App ID and App Secret in the application dashboard. You’ll need these two values in the CloverETL graph.

CloverETL and Facebook Graph API

CloverETL and Facebook

Now we can proceed to CloverETL. This example graph retrieves Facebook friends and stores them in an XML file in alphabetical order.

Facebook Graph API and CloverETL data integration

It uses the HTTPConnector component to retrieve the access token. The query URL is App ID and App Secret are used as HTTP request parameters.

The access token query in the example is:${APP_ID}&client_secret=${APP_SECRET}&grant_type=client_credentials

The response content is:


The access token is parsed using the Reformat component and its value is passed to the next HTTPConnector where it’s used as a query parameter. The query URL is{user_id}/friends – this time with the access_token request parameter. This is a regular Graph API call. See the Graph API documentation for other available methods.

An example query API call:${USER_ID}/friends?access_token=702578739774856%7CB0HwJl3tIFM4ShwBXHEuyB3rB20

Response JSON is parsed using the JSONExtract component and one more HTTP request ({user_id}) is performed for each result (=each friend). This query retrieves detailed information about the friend.

The JSONExtract mapping is very simple because it uses automapping by field names. Mapping used in JSONExtract:

<Mapping element="json_object" outPort="0">

The final part of the graph just sorts detailed info about friends using the ExtSort component and writes them into an XML file using XMLWriter. The XML writer uses this mapping:

<?xml version="1.0" encoding="UTF-8"?>
<root xmlns:clover="">
<clover:element clover:name="facebook_item" clover:inPort="0">
<clover:element clover:name="first_name">$0.first_name</clover:element>
<clover:element clover:name="last_name">$0.last_name</clover:element>

to produce XML like this (real values are replaced by “...” - I do not want to show my friend’s names here):

<?xml version="1.0" encoding="ISO-8859-1"?>


Download the example

Connecting to Facebook using CloverETL - Example

Data integration software and ETL tools provided by the CloverDX platform (formerly known as CloverETL) offer solutions for data management tasks such as data integration, data migration, or data quality. CloverDX is a vital part of enterprise solutions such as data warehousing, business intelligence (BI) or master data management (MDM). CloverDX Designer (formerly known as CloverETL Designer) is a visual data transformation designer that helps define data flows and transformations in a quick, visual, and intuitive way. CloverDX Server (formerly known as CloverETL Server) is an enterprise ETL and data integration runtime environment. It offers a set of enterprise features such as automation, monitoring, user management, real-time ETL, data API services, clustering, or cloud data integration.