January 20, 2015

Akka.NET - One Year Later

Akka.NET - Distributed Actor Model for .NET

I originally posted this at Beyond HTTP, Petabridge’s official blog. Read the original there.

This month marks the one-year anniversary of Akka.NET and it has been a massive year for Akka.NET. In December of 2013, Roger Alsing and I had never met and we were both working on our own actor framework implementations in .NET.

Before it was Akka.NET…

My startup at the time, MarkedUp Analytics, was just starting its R&D efforts to begin developing its real-time marketing automation product on top of our existing in-app analytics product. We knew we needed the Actor model and reliable socket networking software to make the product meet its business requirements, and so I began work on two projects in parallel:

I broke ground on Helios first, right around Thanskgiving 2013. By Christmas Eve I also had a working prototype of Hyperion, who’s actor model implementation was largely powered by the TPL Dataflow library. One weekend in late January, as I was resuming work on Hyperion after taking a break since Christmas, I came across a blog post some guy named Roger Alsing wrote about his port of Akka Actors to .NET.

I cared way more about getting MarkedUp’s in-app marketing product shipped than I did creating my own Actor framework, so I did what anyone in my position would do: furiously port one of Roger’s benchmarks to Hyperion and see if my open source framework was better! Oh wait, that’s a bit of a contradiction. Whatever.

Anyway, I ported the Akka.Benchmark.PingPong example from Roger’s framework, Pigeon, to Hyperion. His benchmarks destroyed mine - like 5x better than my own. Roger had also made a ton of progress on porting areas of Akka that I hadn’t even touched yet such as HOCON configuration and remoting.

So on February, 8th, 2014 I sent him the following email:

Hello Roger!

My name is Aaron Stannard – I’m the Founder of MarkedUp Analytics, a .NET startup in Los Angeles. We build analytics and marketing automation tools for developers who author native applications for Microsoft platforms, including native Windows.

I began my own port of Akka to C# beginning in early December, and took a break right around Christmas. I just got back to it this week and discovered Pigeon when I was researching some details about the TPL Dataflow! I wish you had started this project a few weeks earlier ;)

My implementation of Akka is right around the same stage / maturity as yours, but Pigeon offers much better performance (3.5-5x), is more simply designed than mine, and you’ve already made a lot of headway on features that I haven’t even started on like Remoting and Configuration.

Therefore, I would like to stop working on my own implementation of Akka and support Pigeon instead. I think that would be a much better use of my time than trying to invent it all on my own.

I’m an experienced .NET OSS contributor – I currently maintain FluentCassandra (popular C# Cassandra driver) and have a bunch of projects of my own that I’ve open-sourced.

I plan on using Pigeon in at least two of our services, both of which operate under high loads.

Please let me know how I can help!

Best,

Aaron Stannard • Founder • MarkedUp

P.S. Added a .gitignore for you: https://github.com/rogeralsing/Pigeon/pull/8

And from there Roger and I hit it off, and I worked on Pigeon instead of Hyperion going forward.

Eventually Roger reached out to TypeSafe, the creators of Akka, and gave us permission to use the name for Akka.NET.

The F# Community and Akka.NET

Roger had the idea early on of adding an F# API to Pigeon. I never had much contact with the F# community, even when I worked at Microsoft so I didn’t know what to make of it.

As it turns out, F# support was probably the single greatest decision Roger ever made to boost the popularity and awareness of Akka.NET. The F# community quickly rallied around Akka.NET and even the language’s inventor, Don Syme, got onboard with our mission:

The F# community has a strong culture of contributing to open source and tackling some of the same technical challenges that Akka.NET aims to solve, and many of them began testing, refining, and using Akka.NET.

They rewrote our build system from the RAKE / Albacore-based system that I created to use a much more flexible FAKE F# build system. They rewrote the F# API and started working on other parts of the core Akka.NET engine itself.

Some of our strongest contributors to date have come from the F# community, and I would never have guessed just how impactful these developers would be on expanding the footprint and reach of the project. Without the F# community Akka.NET probably wouldn’t have the level of visibility or community support it enjoys today.

Akka.Remote and Helios

Towards the tail end of March, I had an impending deadline to deliver our marketing automation service for MarkedUp, so I directed most of my energy into bringing Helios into production-ready shape and porting the entire Akka.Remote layer from Akka to Akka.NET.

Porting Akka.Remote to Akka.NET also meant porting finite state machine actors (FSMs), ActorSystem extensions, and ton of work inside Helios. But by mid-April we had most of the guts of Akka.Remote working.

It wasn’t until late June that Remoting ended up being production-ready, after all was said and done. Remoting exposed a lot of little bugs and hitches in things like Ask, ActorSelection, Helios, and more. It took a lot of work from a host of developers to sort it out, but it was well worth it.

MarkedUp was able to deploy its in-app marketing product by late June, and ran Akka.Remote stably in production for thousands of external requests per second at a small load (3% - 15% CPU utilization.)

Akka.TestKit

One of the most impressive feats of engineering on Akka.NET was the porting of Akka.TestKit to C#, a feat Håkan Canberger accomplished pretty much by himself.

Akka.TestKit is a framework used for testing Akka.NET itself, but Akka.TestKit can also be used to test your own user-defined actors too. Porting this to C# required a massive amount of knowledge of the internals of both Akka and Akka.NET, and as well as advanced language features for both Scala and C#.

The TestKit is also supposed to be test framework-agnostic, which means that Hakan had to write separate modules for running the TestKit within MSTest and XUnit. Akka.NET switched from MSTest to XUnit around this time in order to get build server support via AppVeyor, so we had to use both for a period of time and they worked seamlessly.

Part of this project also meant that Hakan had to port some ~300 Akka.NET unit tests from the old, hacky TestKit Roger and I pieced together to the streamlined Akka.TestKit. A big effort in and of itself, but the project is all the more better for it since we can start to match Akka’s test coverage 1:1 now that we’re using the same tools.

Now the Akka.NET team is working on the multi-node TestKit, to help extend Hakan’s work for tests that need to run across multiple Akka.NET actor systems in parallel.

Akka Website and Documentation

In order for an open source project to get any sort of traction, developers need to figure out how to actually use it first. The documentation on Akka.NET’s Github repository was usable only to the most determined hackers.

So Roger decided to solve this problem by creating a beautiful, responsive website that matched most of the documentation from the original Akka project. You can see the finished product here at: http://akkadotnet.github.io/

And it’s made a big difference to Akka.NET’s accessibility to developers who are new to both Akka.NET and the actor model.

Akka.Contrib - Loggers, Dependency Injection, and More

Among the long list of successful concepts we borrowed from Akka is the notion of Akka.Contrib, or contribution areas that aren’t core to the Akka framework itself but are still important to integrating Akka.NET with the rest of the .NET ecosystem.

Most of the contributions so far have been around adding logging, dependency injection, and monitoring. We expect more flavors to come in the future!

Here’s the list, as far as I can recall, of all of the contributions outside of core Akka.NET developers have done this year:

Akka.Cluster Beta

Continuing from the work started on Akka.Remote, David Smith and I teamed up to bring the clustering module to Akka.NET.

The goal behind clustering is to allow developers who want to run Akka.NET in the cloud to elastically add or remove Akka.NET nodes on demand without having to adjust configuration or code in the process. Turn software engineering problems into “throw more hardware at it” problems, and et cetera.

Akka.NET Clustering Documentation

There’s still a lot of work left to be done, but so far we’ve published an initial working implementation of Akka.Cluster. There’s a lot left to fix and test, but the bulk of Akka.Cluster is ported.

F# API Facelift

The tenacity of Akka.NET’s F# team never ceases to amaze me - in November of 2014 Bartosz Sypytkowski, the chief F# contributor on Akka.NET, rewrote the entire F# API for Akka.NET and even had it reviewed by Don Syme himself. Now F# developers can enjoy a functional programming experience on top of Akka.NET that reflects the original Scala and Erlang roots of the Akka implementation of the Actor model.

In addition to writing lots of F# code, Bartosz has been blogging up a storm on how to use the latest Akka.NET features with F#.

Akka.NET Meetups and Community Growth

One of the most exciting parts of getting involved in open source is watching people get interested in what you’ve helped build. So the coolest things we’ve seen over the past year are all of the Akka.NET meetups our contributors and users have hosted to help bring the Actor model to their fellow .NET developers:

Past Meetups

  1. Roger Alsing hosted Akka.NET with the Sweden Progressive .NET User’s group on 9/16/2014 AND Build scalable systems with Akka.NET with the Swenug Developer’s Group on 10/30/2014. (Slides)
  2. Akim Boyko spoke about F# and Akka.NET to the Kiev F# User Group on 12/4/2014. (Slides) (Code Samples)
  3. Anthony Brown hosted An evening with the stars: The actor model and .Net with the Southhampton Developer’s Group (UK) on 1/15/2015. (Slides) (Code Samples)

Upcoming Meetups and Talks

  1. Håkan Canberger is speaking at Swetugg on Introduction to Akka.NET in Stockholm on February 2nd, 2015.
  2. Aaron Stannard is speaking at Cassandra Day LA on DataStax Enterprise and Actor Systems for Scalable, Real-time Decision Making on 2/12/2015.
  3. Riccardo Terrell is speaking at the FP AMS user group on The Actor Model in F# and Akka.Net in Amsterdam on April 8th, 2015.

And most impressively, Roger was interviewed about Akka.NET on .NET Rocks! .

The .NET community embraced Akka.NET in 2014, and we’re going to work together as a developer community to make it even bigger in 2015.

What’s Coming to Akka.NET in 2015

I look back on what the Akka.NET community accomplished in 2014 and I am simply stunned. There were many other contributors than those I named in this post (sorry!) - too many to name. But collectively, we’ve built a platform that serious enterprises can begin to consider for development of microservices, game servers, marketing and e-commerce tools, and even financial trading platforms.

On the horizon for Akka.NET, I am personally hoping for the following in 2015:

  1. Akka.Cluster and Multi-Node TestKit become production ready;
  2. Akka.Persistence becomes production ready;
  3. Akka.ClusterSharding becomes a thing (whaaaaaa);
  4. Full Mono support and documentation;
  5. Built-in monitoring hooks within Akka internals;
  6. And who knows what else the community might invent?

Akka.NET is an exciting project because of the types of developers it attracts - people who already looking outside of the ASP.NET + SQL Server stack as to the solution to every possible problem. The contributors are going to come up with some great stuff this year, and I can’t wait to see what “Akka.NET - Two Years Later” reads like.

Discussion, links, and tweets

I'm the CTO and founder of Petabridge, where I'm making distributed programming for .NET developers easy by working on Akka.NET, Phobos, and more..