Watson Marketing Ideas

Submit new product ideas for Campaign, Interact, Marketing Operations, Marketing Platform, Campaign Managed Hosted, interact Managed Hosted, Marketing Operations Managed Hosted, Digital Analytics, Tealeaf, Universal Behavior Exchange, Watson Customer Experience Analytics, Watson Marketing Insights, Watson Content Hub, Web Content Manager and WebSphere Portal solutions. Before you submit, please review existing ideas; if an idea close to yours already exists, it's better to add comments or vote on the existing idea. We will review your ideas and use them to help prioritize our product development. Best of all, the portal will automatically update you when the status of your idea has been changed.

Submit ideas for other Watson Customer Engagement Products:

•  Watson Campaign Automation
Watson Commerce
Watson Supply Chain

Event Bus Open Source Publishing Agent - Full Data Stream Access

At present the event bus for On Premise is very limited.  The "CEP" agent has three options: File, TCP and HTTP.  However, each of these go through the same CEP transformation, taking fixed fields and mapping them to columns for output.  This means you must know all the fields that are there, in order to extract data.  But if there is other data presented, it will not be captured.

The pipeline Socket agent is proprietary, and requires a handshake to confirm delivery of data downstream, so as such 3rd party software cannot "listen" to it, without it failing to operate, as they do not know the handshake.

This request is to create a more open source agent that will allow ALL the data in the Event Bus pipeline to be real-time data available to other systems, Request and Response payload.  This would allow us to send this data to other messaging systems like Kafka, Streambase, IBM-MQ, Splunk, etc. 

  • Avatar32.5fb70cce7410889e661286fd7f1897de Guest
  • Jun 11 2018
  • Planned for Future Release
How will this idea be used?

Of immediate interest to security, fraud, risk and other that wish to see data and fields that people are trying to inject us with.  Headers, cookie names/values, etc.  We cannot use the Event Bus, as we don't know the fields they are using - that's the point, so we can't pre-state the fields to come out of the CEP agent.  We can't use the Archive agent as it was never updated for high-speed, and file write/read is a waste of system resources.  We need a high-speed in-memory/CPU agent to be able to send the data to a messaging system.

What is your industry? Banking
What is the idea priority? Urgent
DeveloperWorks ID
RTC ID
Link to original RFE
  • Attach files
  • Admin
    ROB HAIN commented
    October 16, 2018 14:23

    Questions:

    • Any input as to the type / types of output required here?  Would JSON and XML be good options to start?  Are there others?
    • Is the sending of ALL data a good option to start?  Or, is configuration required to select a subset of data?
  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    October 16, 2018 16:05

    JSON and XML seem to be valid choices.  I'd also suggest a Kafka producer as well (Java), as this keeps coming up in RUGs, and it's an open standard I believe.

    https://www.oreilly.com/library/view/kafka-the-definitive/9781491936153/ch03.html

    Options for sending the data: Blacklist and whitelist, by sections/fields already accessible by the Privacy agents, the "Section=" and "Field=" lists, like Section=RequestBody

    The one caution I would recommend is making sure multiple of these can run at the same time, on the same machine.  Then someone can run multiple pipeline (SessionRouter agent) copies, with different Privacy filter rules, to different publishers.  Example: Checkout attempt data stream to Kafka, login data stream to Splunk, password change data stream to IBM MQ.