Thoughts on microservices. Part 3: Throwawayware.

Картинки по запросу throw away this

Microservices.. yes, nowadays they are everywhere! In my previous post I’ve shared my thoughts that sometimes just a monolith can do the required task without being cut into microservices. But it is wonderful if microservices are the most suitable solution! This is definitely lovely!

First, we’ve got a lot of benefit it provides to us, second, we’ve got a great technology support to implement them – there are a lot of frameworks and platforms helping us develop some really good microservices.

The most of the conference talks I have recently seen are basically “Hello world” talks. The main message is “Hey, look how easy it is to create a microservice!”, “Just few annotations, and you are up and running!”. How great this is!

But when it comes to real life, most usually between start of the project till the first really useful microservice there is at least one sprint. More often three sprints. You’ll say – “That is ok! We are serious people doing serious enterprise!”. I still ask myself – “What can be so huge in one microservice, that it may require several weeks to be created?”. “Why then this service is called MICRO?”

I’m not sure if there is an official definition of a microservice, but a well-established idea is that it is a piece of software which is designed to perform only one single function, but in the best possible way. It should communicate with the rest of the world with the most lightweight protocol possible, like REST. So, if we have a login functionality, a microservice should be able to login a user in the most fast, easy, reliable, secure way. Only login and nothing else. For logout there should be another service. The size and complexity of such a service are not in scope of micro, they can be of any scale. Technically, there should be only one function exposed, but what happens under the curtains are just implementation detalils.

This is why, the most of such services become really complicated, often even overengineered. From my experience, at the end of the day, each microservice in enterprise webapps becomes a usual three-tier application. It has its “UI” simplified to REST endpoints, it has its business layer with several services interfaces and their implementations, an infrastructure layer to talk to DB and other services. The interlayer communication is done via value objects. As a result, a usual call of a function of this microservice ends up with several data coping from one value object to another (in one of such microservices I’ve seen up to 14 of such hops). Why would you do this? The typical answer is if some part of the service changes, we will change only this part of it. Fair enough. If in return I ask, how many times in your life have you ever changed one of the tiers without affecting the others? Тhe answer is usually – never.

Once again, nothing wrong here. The complexity of the implementation of a microservice is orthogonal to the interface it provides.

Another question: how many times have you ever received a fully functional “freezed” interface definition? The spec that never changes? A guaranteed never to change document? In my case – never!

The changes are coming constant or even increasing speed. Even in terms of one API version. You may say: You are doing it wrong! You should plan more carefully, you should rethink your spec versioning policy etc. And you will be completely wright!

But.. there is something called real life! There are business needs that require quick reaction, quick change, quick time to market!

Looks like this constant change is just inevitable. For my 13 years in software development there were practically no projects that had really smooth stable development. A least every sprint there were some “tiny” spec changes, causing all codebase to suddenly go red. I’m sure I’m not the only “lucky” having this.

How should we solve this kind of problems in the microservices world? Its quite time consuming to create beautiful, complex, nicely architectured, (at least) three-tier microservices without frozen spec. As I already said, those spec changes usually require changes in all of the layers. This is hard! Most of us thought that’s just the way it is. Software development is hard.

But this situation kept me bothering. May be there is a way to expedite this change reaction? May be a microservice should be micro in all aspects? I mean really tiny? With as less internal abstractions as possible? Really tight coupled inside? When I’m writing this, I have a feeling I’m breaking the law. It is like I’m cancelling everything I’ve learned from the CS courses in the university. Why don’t we throw away all of the … tiers? Make it a really tiny one-tied micro monolith? A microlith? So that the object or even a direct line with data we receive from the infrastructure we transform to JSON object in one single class? With all the business logic also included in this one and the same class? You are absolutely eligible to say – “are you nuts?” What if the infrastructure changes? What if the Rest API changes? What if the business logic changes? You have to rewrite everything!

And then I suddenly say – “Yes! We will rewrite this service from scratch!”. Luckily, a lot of code in the service can be just generated. So, yes. We will rewrite it from scratch! We copy/paste. We do all the possible anti patterns. Only to make it work according to the spec and pass the tests! “But isn’t it a lot of effort?”. My answer is – “Pretty much the same as changing all of the classes in all of the three (or more) tiers!”. And people usually say – “Hmm…”.

This may sound really strange, but through several last projects, writing some really ugly small one tier microservices with really almost no architecture and no refactoring, then rewriting this from scratch saved me a lot of efforts. I have even created a name for this piece of software – “Throwawayware!”. Try to say it quick!

The next question that usually comes is – “Do you put this ugly thing in production??”.

My answer is: – “Ok, you got me! No, this does not go to production! May be sometimes..”. What actually goes there?

Putting this ugly little thing in production will be just catastrophic. That’s why I’ve tried some mixed approach.

First of all, we usually develop the tests to follow the contract. Yep, although I’m not a big fan of TDD, I think this is a good place to use it. Adjusting the tests to follow the spec and the contract is always the first priority to us!

Then we usually have 4 to 10 iterations on each microservice in this ridiculous “throw away” way. Funny thing is that up to 50% of the services do not even survive through those iterations! At some point the service may come obsolete even before production. This means that we usually save a lot of efforts for not creating something complex for something that will never be used (“Bingo!”).

Then when the service gets through those several iterations, we consider this service as “survivor”. At this time usually, the spec stabiles as well. And we do the same for each API version.

Just like the Garbage collector in JVM.. Haha! Yep, like the GC! Lovely!

For the survivors we usually make some really complex refactoring, or rewriting it completely new. We try to make their code really readable, expandable and maintainable. Quite often from the usage of those “throw away” implementations we can see some before unpredicted usage cases, like for example the pressure it should hold, or security, or fault times etc. This may end in three-tier architecture, but with really nicely designed layers.

Looks like this “two stages approach” works really well! Funny thing is that it fits perfectly in our SCRUM cycle. May be, I’m wrong, but this approach saved us a lot of effots and helped us establish good quality implementations in shorter time. In three projects so far.. We’ll see how it goes!

Any critic is welcome though…

TRIP report: Codefest. The siberian adventure. 🇷🇺

Right after Joker 2018 I was approached by one of the speakers Ivan Ugliansky with a very interesting proposal – “Hey, you know I’m from Novosibirsk, we have quite a big conference named CodeFest happening usually in the end of March. Would you like to come to Siberia?”. I answered – “This will be an honor for me!”.

It took me about 12 hours to reach Novosibirsk from Sofia. First 3h30m from Sofia to Moscow. Then I had my shortest night for this year – we took off at 21:30 MSK and landed 5:30 NSK time. The time passed quickly, the Aeroflot flight attendants were we so beautiful, that all of my attention mainly focused on them.

When we landed in Novosibirsk, I just couldn’t stop staring at my GPS position. Wow, I’m almost in the middle of Eurasia!

No photo description available.

The first day was quite intense! My great friend Ivan has made a wonderful excursion to a place named “Akademgorod”. In English this may sound like “academics village”. It was created from scratch bak in the 60s to accommodate Soviet workers of science in a very peaceful close to nature environments. As a result it has transformed in a huge scientific cluster. A lot of institutes and research laboratories are now located there. There is also a huge university.

Students and specialists from all over Siberia and even Russia come to this place for study and cool projects!

No surprise, there is a big IT presence here. A lot of Russian and not only Russian companies have their R&D here, and no surprise the CodeFest conference gathers more than 3k attendees.

I had the privilege to be invited to the office of JetBrains

and the office of ExcelsiorJET

I had some truly wonderful time there! We had some pizza and great chat! Huge thanks to Ivan for being such a great host.

The end of the day was dedicated to a special event – the local JUG meetup. Together with Paul Finkelstein I had the honor to give my talk about Java and JPU in front of the local community. The event took place on the 22nd floor in the office of the famous Russian company 2GIS. The location was really awesome!

More than 150 people come to the event! It was fully packed with even standing people!

To be honest, I was a bit scared to be in front of these people. Siberian developers are famous to be the most hardcore developers in the world. Mostly the strongest compiler writers. Isn’t my talk going to be too smoothy for them? At the end of it I was happy to find out that it was really ok! I had a lot of questions afterwards!

It was a sincere pleasure to talk to this community!

Картинки по запросу novosibirsk expo center

The next day was the first day of the CodeFest. Luckily I had my talk scheduled for the day two. It was really good news for me, since I still had some jetlag.

So.. the conference is huge! It has 7 tracks and it is not focused only on Java.

*Siberia is here!!

Actually the variaty of technologies and a SC fields was a big plus for me. I learned a lot of new stuff. I think I was mostly stuck at the Data Science track. I’ve been playing a lot with tensorflow lately, so it was a natural choice for me.

I’ve heard some nice Project management and Team lead oriented talks as well.

I was even interview on the conference radio. I had a lovely chat with Vladimir Plizga from CFT.

The conf is huge!

The day one ended in the local pub with the concert of … the speakers! Singing about.. coding!! Awesome experience!

My talk was scheduled to be the first on day two.. I’ll be honest, I have expected no more than three people to show up. Three listeners would have been a victory for me!

Surprisingly.. it was almost packed for my Microprofile.io talk! (based on the MicroProfile.io tutorial created together with Ivan. St. Ivanov)

Every went smoothly! All of my demos worked! What a releaf!

Usually its not just giving a talk on CodeFest and you are free. After the talk there is an expert zone, where the speaker can make some live discussions and answer audience questions. I spent almost an hour there! I’m really happy that Microprofile.io enjoys so much interest! I hope it was useful for the audience!

The second part of the day I have decided to spend in the city exploring it.

The famous Transsib railway is crossing the big Siberian river Ob here.

The old and new come together!

The cultural life is very intense!

The famous Opera and Ballet theatre is really huge!

Siberia and Novosibirsk are definitely wonderful places to visit! If you have the chance to go there, you definitely should!

I’ve met really wonderful people and had some amazing time!

The next day I was in the airport having something I’ve almost missed – Siberian pelmeni!

Image may contain: food

Ok, I’m going home! Siberia, you are endless and beautiful!

Image may contain: sky, airplane, cloud, outdoor and nature

Yet another 12000 km up in the air! (SOF-SVO-OVB-SVO-SOF)

Huge thanks to Ivan Ugliansky for making this possible.

Thoughts on microservices. Part 2. Evolution of Microprofile.io

Looks like public transport is a good place to blog. For several hours I just have a time on my own. I may stay focused on the topic.

So, I’m now on my flight from Novosibirsk to Moscow after wonderful CodeFest conference! I’ve got four free hours, an A/C plug and no free internet (yes, there is payed one). Only the views of immense Siberia from my window .. and the beauty of Aeroflot stewardesses disturb me a little bit when passing by…

I had an amazing time in Siberia! I had a lot of really exciting moments there! I met a lot of truly interesting people and had the pleasure to talk to them! Trip report to CodeFest will be published soon!

But this time on the plane I have I want to dedicate to some other thoughts I recently had in my mind, and they are about the evolution of MicroProfile.io.

I had the privilege to be invited to CodeFest to give my talk right about MicroProfile.io in Russian. I was very happy that the audience was so excited about this project. I had a lot of questions that kept me in the discussion zone for more than one hour after my talk. The interest was great!

The MicroProfile.io project is an amazing example of a community work! It is very young, but it evolves extremely fast! I was the eyewitness of the birth of the project, when it has started with just three specs inside CDI, JSON-P, JAX-RS. It was kind of brave move to start with just these specs, but the same time it was very wise IMHO. This ignited a lot discussion and community work. Shortly after that a brand-new Config spec was created, allowing us the creation of not only portable code, but portable configuration as well.

Community worked hard to establish what is really useful in the microservices world and created Fault tolerance, Health checks, Metrics and JWT propagation. The specs like OpenAPI, RestClient and OpenTracing made the picture complete. JSON-B came as a cherry on the top. Every new version of the MicroProfile.io came with some new specs and evolutions of the already available ones, definitely great! Let us not forget that all this happened only in about two years period.

Every time together with Ivan Ivanov we gave our workshop on MicroProfile, or we gave our talks about it we saw some really huge enthusiasm in the eyes of our attendees. Finally, there is a lightweight absolutely standardized and portable way to write microservices in Java world.

The platform vendors from their side made some really tremendous efforts to implement all of those specs. Some of the servers were rewritten like from scratch. Completely new projects like Helidon and Quarkus emerged.

Starting from MicroProfile version 1.2 (which I believe is mature enough) I’ve started using it in my projects. Although it is a bit hard with the testing, I’m quite happy about it. I just love standard code, I like the portability. I even have used it twice, when we had to switch servers from different vendors. It really went seamlessly. In one of my previous post I’ve described how easy it was for us to migrate to Quarkus one of our services. Almost copy/paste! The power of standards!

There are some things that nevertheless bother me. I’m reading carefully the discussion in the MicroProfile.io mailing list and I’m not sure if, for example, adding a persistence spec to the project will be a good idea. I know, a lot of services are stateful and persist a lot… but is it really relevant to microservices? Everything can persist and this is not something specific to microservices. Why then should we put it to the MicroProfile.io project? There are a lot of stateless services. They don’t need any persistence.

I had the pleasure to share these my thoughts with Emily Jiang at Voxxed Bucharest. I have proposed to start thinking a little bit differently – why won’t we make separate profiles relevant only specific problem we are solving? Should we have a special “Persistence profile”? Or if we want to make thing reactively we should establish a “Reactive profile”? And if we need more than one profile, we just combine them.

This may sound even absurd, but I believe, that current set of specs is just enough for MicroProfile.io. There are already 12 of them. Most of them are really microservices specific like Fault tolerance etc., other like CDI are the foundation. Should JPA spec one of the foundation specs? I really don’t think so. If CDI is really a glue to put it all together, JPA can be sometimes totally ignored, if the service is stateless.

I’m still not sure if I’m right in my thoughts. I sincerely wait for critics. But I believe that staying focused on microservice relevant specs is a better evolution path. There are more microservice specific thing to be discussed, like better servicemesh integration. Or a common set of server flags?

Currently these 12 specs in MicroProfile 2.2 solve all of my microservice specific tasks. It’s only the JSON-P spec that I’m not using it anymore, I do it all through JSON-B 🙂

Ok, our plane is approaching Moscow, I already see the Ostankino tower and the Moscow-city skyscrapers. Time to switch off the laptop..