SegBITS Protocols: Using Transformations to fix bad data

Good move calling it as a “band-it”, still, this is a nice feature. I am curious about the duplicated events you mention. How do you use this feature to address them?

Hi @Andrew Ferreira, great question! In this case the Duplicated Events I was referring to were events that serve the same purpose for the organization but have different naming conventions. For example, if a user were tracking Page View events as both Page View (title case) and pageView (camel case). 

In this instance, our tracking plan states that the default casing convention should be title case, so we would build a transformation that transforms the pageView event in-flight to Page View to properly align to our tracking plan. This ensures that our destinations are receiving the event in the proper format to be used by whichever teams and tools are connected and in a format that is consistent with what they would expect. 

Does that help to provide additional clarity?

It does. So in this case, Segment would receive the original Page View and another Page View (the transformed version of pageView), right? Would Segment recognize the duplicated data and drop one of them?

Assuming that you are ingesting both the Page View and pageView events into Segment as unique events in different casing conventions, these would be considered duplicate events. If you leverage a Transformation to transform pageView into Page View this would still result in two unique Page View events as Segment will not automatically drop one of them. Therefore the true solution (for two unique events) would be to go into your codebase and eliminate the duplicated event in the non-desired format, resulting in a single event in the desired casing convention. 

A Transformation is a mechanism to transform a single event’s naming convention in-flight in Segment prior to reaching its connected Destinations. So if you have a single event pageView coming into Segment that you’d like to transform into Title Case you can create a transformation to do so, reformatting upon ingesting the event into Segment and before it’s sent to the connected downstream Destinations. 

The more popular use case for Transformations is to transform events in-flight to satisfy the requirements of a Destination that can only accept events in a specific format. 

Apologies for any confusion my initial example may have caused, hopefully this helps a bit more. Thank you for the thoughtful questions! 

I appreciate the simplicity of Protocols transformations, but in my experience the necessary transformation requires something more complicated and usually results in a custom function and then an engineering ticket 😀