Back to CloverDX Blog on Data Integration

Deploying a custom component plugin to the CloverETL Server

Posted by Jan Sedláček on December 16, 2013

A few weeks ago, we published a blog post about how to create a custom component. It described all of the necessary steps to implement your own component and how to integrate it into the CloverETL Designer. However, if you use the CloverETL Server and want to be able to use your component there, you need to go one step further. In this post, I'll show you how to deploy the plugin into the Server so that your components also work in transformations living on the Server.

For the purpose of this demonstration, I'm going to use the component we created in the previous blog post. You can download the project template (you will only need the project com.cloveretl.customcomponent) or follow steps 1 through 9 to create your own component plugin. Please note that you only need the "engine" core plugin for the Server. For now, you can ignore steps 10-23 which are only needed to make the component usable in the Designer. However, in the future, you'll always want to deploy your components into both.

Once you have the plugin ready, pick a location on the Server machine where your active plugins will live. The location must be readable by the CloverETL Server process. Also, I recommend choosing a place outside the Server installation folder.

Now, let's configure the CloverETL Server. It's fairly easy. For this example, I'll use the config file [tomcat_home]/conf/Catalina/localhost/clover.xml. Although, keep in mind there are several other ways to configure the Server.

The engine.plugins.additional.src config property specifies the location you chose for your plugins. Each plugin located there has to be in its own directory.

Simply add this XML element to the config file:

<Parameter name="engine.plugins.additional.src" value="[Path-To-Your-Engine-Plugin-Location]" override="false" />  

After saving the changes, you need to restart the CloverETL Server. From now on, you should be able to use the new components in the Server.

To verify that all your plugins are available in the Server, log into the Server Console and go to Configuration > CloverETL Info > Plugins. You should find something like this:

custom component plugin

Upon startup, the Server deploys all the configured custom plugins into its working directory. Every time you restart the Server, all plugins from the working directory are removed and redeployed. Thus, if you remove your custom plugin from the configured path above, it will also be removed from the Server working directory after restart.

If you chose to use the sample plugin from our previous blog post, you can take the HelloWordTest project, put it to your Server and execute testHelloWorld.grf. It should run successfully and you should find this output in the log in Execution History of the graph.

2013-12-16 09:49:56,800 INFO  1703936 [TRASH_1703936] +-------+------------------+-------------------------+------+
2013-12-16 09:49:56,801 INFO  1703936 [TRASH_1703936] |Record |name              |city                     |age   |
2013-12-16 09:49:56,802 INFO  1703936 [TRASH_1703936] +-------+------------------+-------------------------+------+
2013-12-16 09:49:56,802 INFO  1703936 [TRASH_1703936] |# 1    |Hello Sonya       |Hello Ottawa-Carleton    |49    |
2013-12-16 09:49:56,802 INFO  1703936 [TRASH_1703936] |# 2    |Hello Fleur       |Hello Newtonmore         |26    |
2013-12-16 09:49:56,803 INFO  1703936 [TRASH_1703936] |# 3    |Hello Addison     |Hello Pictou             |58    |
2013-12-16 09:49:56,803 INFO  1703936 [TRASH_1703936] |# 4    |Hello Ariel       |Hello Plymouth           |40    |
2013-12-16 09:49:56,803 INFO  1703936 [TRASH_1703936] |# 5    |Hello John        |Hello Cheyenne           |24    |
2013-12-16 09:49:56,803 INFO  1703936 [TRASH_1703936] |# 6    |Hello Hasad       |Hello Deschambault       |61    |
2013-12-16 09:49:56,803 INFO  1703936 [TRASH_1703936] |# 7    |Hello Abel        |Hello Roosbeek           |23    |
2013-12-16 09:49:56,804 INFO  1703936 [TRASH_1703936] |# 8    |Hello Xyla        |Hello Rycroft            |37    |
2013-12-16 09:49:56,804 INFO  1703936 [TRASH_1703936] |# 9    |Hello Keelie      |Hello Matlock            |79    |
2013-12-16 09:49:56,804 INFO  1703936 [TRASH_1703936] |# 10   |Hello Trevor      |Hello Rodengo/Rodeneck   |70    |
2013-12-16 09:49:56,804 INFO  1703936 [TRASH_1703936] |# 11   |Hello Yardley     |Hello Diepenbeek         |26    |
2013-12-16 09:49:56,805 INFO  1703936 [TRASH_1703936] |# 12   |Hello Steven      |Hello Werbomont          |43    |
2013-12-16 09:49:56,805 INFO  1703936 [TRASH_1703936] |# 13   |Hello Patience    |Hello Kirkwall           |43    |
2013-12-16 09:49:56,805 INFO  1703936 [TRASH_1703936] |# 14   |Hello Charlotte   |Hello Labro              |47    |
2013-12-16 09:49:56,805 INFO  1703936 [TRASH_1703936] |# 15   |Hello Ashely      |Hello Madrid             |32    |
2013-12-16 09:49:56,806 INFO  1703936 [TRASH_1703936] +-------+------------------+-------------------------+------+

Data integration software and ETL tools provided by the CloverDX platform (formerly known as CloverETL) offer solutions for data management tasks such as data integration, data migration, or data quality. CloverDX is a vital part of enterprise solutions such as data warehousing, business intelligence (BI) or master data management (MDM). CloverDX Designer (formerly known as CloverETL Designer) is a visual data transformation designer that helps define data flows and transformations in a quick, visual, and intuitive way. CloverDX Server (formerly known as CloverETL Server) is an enterprise ETL and data integration runtime environment. It offers a set of enterprise features such as automation, monitoring, user management, real-time ETL, data API services, clustering, or cloud data integration.