Oracle SaaS Extract Using BICC

In my previous post Downloading files from Oracle UCM in Python I showed how calls can be made to Oracle WebCenter to download files in Python. There is another Java based utility which can abstract these SOAP calls and help us achieve the same. This utility is called Document Transfer Utility. I will share this method in my future posts.

Now, this is an adhoc or batch processing where we trigger this python or java based script to go fetch a file by name or search through an UCM directory and download its contents. For this, you need to know the file name, document ID or document name. And secondly, you need to know if the file has been generated by a BICC schedule. BICC(Business Intelligence Cloud Connector), is an extract mechanism to handle robust and bulk data exports. BICC can be configured to write the extracted data files to UCM or Cloud Object Store. So we need a way to listen to BICC events knowing when the extract has been triggered, and completed successfully. Unfortunately all Oracle SaaS Events are not directly exposed to outside Oracle Cloud world.

Still there is an option to go through OCI. OCI, Oracle Cloud Infrastructure provide Events & Notifications as part of integration tools. We can trigger an event as soon as a file has been written and Notification integration service can apprise the consumers on the event and provide the details of the file(s) that is written.

In the above diagram Batch processing can be done by asking BICC to write to UCM and then we do a pull from non-Oracle Cloud env.. Both BICC & UCM have no cost associated.

Instead of doing adhoc or batch processing if you wish to respond to an event and pull the data as soon as it has been extracted then you can consider Near Real-time path to subscribe to OCI Notifications and use Rest Services provided by OCI Object Store REST services to pull the files.

So in effect we will be using below services to achieve our goal,

1. BICC, to extract data
2. UCM or Object Store where BICC writes the extracted data in compressed csv format
3. OCI Events Service to trigger an on a successful BICC extract
4. OCI Notifications Service to notify of the event
5. OCI Object Store REST services/ UCM SOAP services to download the files

The consumer can be on-premises or in another cloud with a message broker that is subscribing to the notifications from OCI and Stream Processor to process the files.