I love Integration/Middleware space. In the spring of 2004 I was working on a large implementation for a client. We had to integrate externally, internally with a large number of systems and we also had a need for long running processes. We were already using BizTalk 2002. We came to know about a radical new version of BizTalk Server called BizTalk 2004. It was based on .Net and was re-written from scratch. As soon as we learned about its capabilities we knew that it was a far better product for what we were implementing. We made a decision to use BizTalk Server 2004 Beta during our development. Since the product we were building was releasing in fall/winter we knew that it will become generally available before we go live. Making the decision to switch to BizTalk 2004 was an easy decision. Hard part came when I had to design 30 plus long running processes using BizTalk. There wasn’t any documentation. There were no BizTalk experts we could reach out it. At that time somebody began publishing a guide called “Bloggers Guide to BizTalk”. It was a compiled help file which included blog posts from authors all over the world. Without this guide we would have failed to implement our solution using BizTalk 2004.
I still like middleware space but I have added Cloud, IOT, DevOps to my list of technologies I use every day. Azure Event Hub is a relatively new PAAS Service that was announced in last Build conference. It became Generally Available at TechEd Barcelona in October 2014. I will use this blog post to document various resources about Azure ServiceBus EventHub service. I named it “Bloggers Guide to Azure Event Hub” as an Ode to “Bloggers Guide to BizTalk”. I want to make it easier for anybody learning about Azure Event Hub to find helpful resources that will quickly get them started. I will make weekly updates to keep it current.
Introduction to EventHub from TechEd Barcelona: http://channel9.msdn.com/Events/TechEd/Europe/2014/CDP-B307
Cloud Cover Show about Event Hub: http://search.channel9.msdn.com/content/result?sid=b8411351-e4b2-4fff-bb3c-a64b566c7d99&rid=85437dcd-37ee-4965-ab09-a3d4013c30d7
Event Hub Overview: https://msdn.microsoft.com/en-us/library/azure/dn836025.aspx
Eventh Hubs Programming Guide: https://msdn.microsoft.com/en-us/library/azure/dn789972.aspx
Event Hub API Overview: https://msdn.microsoft.com/en-us/library/azure/dn790190.aspx
Event Processor Host
EventProcessorHost class: https://msdn.microsoft.com/en-us/library/azure/microsoft.servicebus.messaging.eventprocessorhost.aspx
EventProcessor Host is covered in the API overview but I want to call this out once again as it is the easiest way to process messages out of Event Hub. It may meet the needs of more 90-95% of scenarios. To get an in depth understanding of EventProcessorHost you should read this series of blog posts from Dan Rosanova.
Event Processor Host Best Practices Part I : http://blogs.msdn.com/b/servicebus/archive/2015/01/16/event-processor-host-best-practices-part-1.aspx
Event Process Host Best Practices Part II: http://blogs.msdn.com/b/servicebus/archive/2015/01/21/event-processor-host-best-practices-part-2.aspx
ServiceBus Event Hubs Getting Started : https://code.msdn.microsoft.com/windowsapps/Service-Bus-Event-Hub-286fd097
Scale Out Event Processing with Event Hub: https://code.msdn.microsoft.com/windowsapps/Service-Bus-Event-Hub-45f43fc3
ServiceBus Event Hub Direct Receiver: https://code.msdn.microsoft.com/windowsapps/Event-Hub-Direct-Receivers-13fa95c6
If you are looking for reference architecture and code sample for how to build a scalable real world application data-pipeline will be helpful to you.
Real-Time Event Processing with Microsoft Azure Stream Analytics
This reference architecture is about Stream Analytics but it shows how Event Hub is a core part of the real-time event processing architecture.
Anybody developing ServiceBus application should be using this tool. It has Queues, Topics and EventHub support as well.
If you want to provision EventHub in Azure you options are:
1. Use the Azure Management Portal
2. Use the SDK to provision it in code
3. Use the REST API
4. Paolo Salvatori created a PowerShell Script that invokes the REST API to create Service Bus namespace and EventHub. This is the script I am using my current project. http://blogs.msdn.com/b/paolos/archive/2014/12/01/how-to-create-a-service-bus-namespace-and-an-event-hub-using-a-powershell-script.aspx
EventHub makes an excellent target for ingesting logs at scale.
Serilog is a easy to use .Net structured logging framework. It already has an EventHub Appender. You can check it out here:
Miscellaneous Blog Posts
Azure Event Hubs – All my thoughts by Nino Crudele: http://ninocrudele.me/2014/12/12/azure-event-hub-all-my-thoughts/
Getting Started with Azure Event Hub by Fabric Controller: http://fabriccontroller.net/blog/posts/getting-started-azure-service-bus-event-hubs-building-a-real-time-log-stream/
Azure’s New Event Hub – Brent’s Notepad: https://brentdacodemonkey.wordpress.com/2014/11/18/azures-new-event-hub/
Sending Raspberry Pi data to Event Hub and many blog posts about Azure Event Hub on Faister’s blog: http://blog.faister.com/
Sending Kinect data to Azure Event Hub at Alejandro’s blog: http://blogs.southworks.net/ajezierski/2014/11/10/azure-event-hubs-the-thing-and-the-internet/
Azure Stream Analytics, Scenarios and Introduction by Sam Vanhoutte. This blog post is about Azure Stream Analytics but both of these services will work together in many scenarios. http://www.codit.eu/blog/2015/01/azure-stream-analytics-getting-started/
Azure Event Hub Updates from a NetMF Device on Dev Mobiles blog: http://blog.devmobile.co.nz/2014/11/30/azure-event-hub-updates-from-a-netmf-device/