Kafka in the Cloud: Why it’s 10x better with Confluent | Find out more
Transportation providers have long relied on manual processes to keep their operations flowing. Physical records, filing cabinets, and customer service agents have enabled them to transport people from Point A to Point B safely and efficiently. While manual processes have underpinned the operations of these companies for years, they’ve also held them back from scaling. This is because today’s travelers expect a seamless digital experience, similar to their favorite mobile apps (Uber, Lyft etc.). They want to be able to book transportation with the click of a button, with clear and accurate pricing, clear arrival/drop-off estimates, and a seamless online booking experience.
Busie, a New York-based company and finalist in Confluent’s inaugural Data Streaming Startup Challenge enables ground transportation providers to provide this kind of modern experience to their customers. Their Charter Platform offers a plug-and-play, modern solution which allows ground transportation providers to provide an end-to-end online booking process, delighting charter customers, reducing customer acquisition costs, and ultimately doubling booking conversions. Transportation providers can digitize and streamline their operations, covering everything from omnichannel quotes, inventory management, and dynamic pricing to payments, dynamic vehicle-based route planning and customer analytics. At the foundation of their platform is data streaming with Confluent Cloud.
Read on to learn how Busie uses data streaming to bring digital services to transportation providers, and why they decided to build their platform on top of Confluent Cloud.
Busie’s co-founders, Brady Perry and Louis Bookoff, made the decision to implement an event-driven architecture for their charter platform early on. They had experimented with a monolithic infrastructure design during their initial product development but found it limiting; the tightly coupled system would prove difficult to scale to the throughputs they expected and would hold back the rapid development of new features on the platform.
While their initial microservices architecture was heavily based on RESTful API calls, they recognized the importance of laying a foundational data infrastructure which could scale and flex to changing needs over time. Having been introduced to the concept of data streaming via Confluent’s developer resources, they knew Apache Kafka would meet these needs. As a lean startup, though, they lacked the resources and expertise to self-manage Kafka effectively, so they applied to Confluent’s startup program which provides up to one year of free Confluent Cloud credits and access to internal experts for early-stage startups with innovative data streaming use cases.
With Confluent, Busie could take advantage of extensive developer tooling (e.g., multi-language support) to quickly onboard new engineers and build a durable event-driven architecture to power their charter platform from the get-go. They were also able to take advantage of the comprehensive fully managed resources that Confluent Cloud provides, including fully managed connectors, stream processing, and Schema Registry.
Busie’s data streaming journey began with unlocking data incrementally from siloed databases. They leveraged a number of pre-built connectors, both fully managed in Confluent Cloud and self-managed on AWS ECS with Fargate, to connect data from 13 PostgresSQL and MongoDB databases (fig.1). This broke down silos between the databases and unlocked them for consumption by new downstream applications.
Having created a foundational data streaming architecture, Busie began creating applications that leveraged further features of Confluent Cloud’s data streaming platform (DSP). Their core application, Range Operator Portal (a verticalized ERP for the end-to-end charter booking process), for instance, is powered by a robust search engine called the ‘Charter Experience Finder’ (ChEF).
ChEF takes a shift-left processing approach, aggregating and enriching data from disparate data sources within Confluent Cloud in order to create a unified data model optimized for search, pagination, and filtering. This enables transport operators to quickly find the information they need. The time taken to load the core view within the Range Operator Portal has, in fact, reduced by nearly 300%, providing a more seamless digital experience to customers—all as a result of this shift left of data processing.
Busie’s customers can also perform powerful search operations on their data, getting them to the information they need faster than any other product on the market, saving them valuable time in their process. Beyond this, Busie has been able to provide their customers with features never experienced before in the industry: dynamic pricing, asynchronous online invoicing, e-contracts and real-time notifications (e.g., on charter status) are just a couple of the new innovations that are made possible by data streaming.
Importantly, Busie is able to roll out such features within weeks or months rather than years (or not at all) as a result of Confluent Cloud’s extensive tooling. They’re able to easily integrate new sources of data into the platform and create well-structured, governed Data Products that are easily discoverable and ready to be consumed by downstream applications.
As a lean start-up with limited resources, this has proved invaluable in responding quickly to the demands of their customers and developing new functionality on the charter platform to the point where Busie is able to support quote volumes for their customers in the millions of dollars per month. It’s also enabled them to better control data storage costs. Confluent Cloud’s stream processing allows them to effectively filter streams of data, limiting the volume of data reaching downstream databases to only what is strictly necessary.
As Busie continues to innovate, they plan to further utilize the capabilities of Confluent’s Data Streaming Platform. Stream processing with fully-managed Apache Flink, for example, presents the team with a number of opportunities to develop their product offering. Real-time analytics dashboards, yield-based pricing (i.e., dynamic pricing based on driver/vehicle availability), and real-time vehicle location reporting are just a few of the streaming applications under consideration.
Data streaming is fundamental to Busie’s mission of helping transportation providers go digital. With Confluent Cloud as the backbone of their Charter Platform, Busie is able to provide a plug-and-play solution tailored to transportation companies which have long been overlooked by tech companies. The streaming, processing, governing and sharing of real-time data from multiple sources allows Busie to offer a comprehensive platform which manages everything from quoting to dynamic route planning.
And with Confluent’s support, Busie is able to develop their product at pace, driving innovation quickly and with limited resources. Having made the early decision to base their platform on Confluent Cloud, Busie has paved the way for long-term success.
This blog explores how cloud service providers (CSPs) and managed service providers (MSPs) increasingly recognize the advantages of leveraging Confluent to deliver fully managed Kafka services to their clients. Confluent enables these service providers to deliver higher value offerings to wider...
With Confluent sitting at the core of their data infrastructure, Atomic Tessellator provides a powerful platform for molecular research backed by computational methods, focusing on catalyst discovery. Read on to learn how data streaming plays a central role in their technology.
