Skip to content

Micro-services – a year after

Wow, I feel like I am wandering into this huge field with landmines right now. This discussion has many, many topics and I will only focus on some of them.

Our experience

MICROSERVICE STRATEGY is kinda like something everyone in Tech has heard of and everyone wants to do, right? And we have now experienced some of the pain that comes when you split up the monolithic software:

  • Overhead when setting up the pipelines
    • Our Teamcity, Octopus and F5s must be configured correctly
  • MANY dependencies to keep track of
    • A client is suddenly not able to spin up locally without 20 other services
  • MANY variables when deploying to different environments
    • Since a service has lots of dependencies, those dependency endpoints are currently defined in each services’ web.config, and those values differ dependant upon which environment deploy to

Our deployment server has currently defined 153 different deployment of services, and they keep rising.

Did we forget the database?

I believe that we are doing the right thing, breaking up those monoliths.

However there is something missing in our current strategy. Although the number of services are still rising, the number of databases is not. Our company has had a long-lasting policy with 1 database pr customer.

It is said that microservices (or whatever you want to call it) are bounded contexts, and they should own their own storage. Still, none of the services created the last years (except SAAM) has gotten their own storage.

Now I am not saying that every service should have their own storage, but it makes sense on a larger level, the application domains.

From a infrastructure perspective, this makes life less flexible. Let’s say you want to move an application to another datacenter. In our scenario you cannot do that, because the database the application uses is also used by many other applications. It’s also insanely more easier and flexible to deal with backup/restore, data retention.

And from a developer point of view, those who want to have more control of the database, and introduce version control and continuous database delivery: it is much easier to start with a smaller database than with the WHOLE thingy.

We already have a ConnectionString service that handles mutliple customer databases, and when I asked all tech-employees in my company a year ago what was our best improvement, this was it. Still, there hasn’t been any initiatives on the database-split approach from the product teams. I guess it’s hard to get priority on such tasks when competing with other features that gives more direct value to the customers.

ms

Service DiscoveryBilderesultat for consul sidecar

While the overhead of defining endpoints for each web.config in every service is getting larger, this certainly slows down the progress on improving our environments. It is clearly not a good idea having to know every endpoint (hostname/ip and port) out there, and it was certainly not a good idea of just having backend services as IPs only (you have to remember that 10.151.36.204 was ServiceA-Prod and 10.151.36.206 was ServiceA-Pilot).

I think some kind of service register and service discovery solution is needed and I believe it’s ok to start with the downstream services, the backend and not the clients. We also have to keep in mind that our company is an enterprise with different datacenters, multiple acquisitions and different hosting solutions, so a discovery solution must be really flexible. One scenario could be introducing Consul (or a similar sidecar concept) on our VMs. Consul will also work with Kubernetes and container solutions (when that time comes).

This concept will totally eliminate all those variable plumbings in Octopus, because when you want to call a service, all you gotta do is to call the local proxy with the name of the service. (like http://localhost:9999/ServiceA). All hosts will have a proxy and they are all in sync all then time. And the proxy is isolated to 1 environment so you don’t have to worry about calling ServiceA-Production or ServiceA-Pilot.

With this approach we can start small with getting our proxies installed on the hosts, and then start calling 1 service. After stabilizing we can start industrializing it more and more.

Introducing New Relic infrastructure

We have now taken New Relic Infrastructure into use.

This a natural improvement of the New Relic Server Monitor tool, with more metrics and aggregations.

Alerting of Windows Services

The best thing is that we now have monitoring of our windows services! This has to bet set up manually per node, but in the very early future, Octopus Deploy will automatically create these alert conditions on deployment of windows services.

Alerting policies

Each team will have their own Alert Policies and alert channels.

This means that very soon it’s not the Application Management team that receives a notification like ‘Error Rate > 5% on application Hours’, it’s the team that owns the service. And the team must act on these alerts themselves.

This is a really helpful tool for monitoring, health checking and debugging our application and hopefully it will empower our teams.

For more information about New Relic Infrastructure, visit https://newrelic.com/infrastructure

Resurrection

Hello again!

After many years and several jobs and several other blog attempts, I have decided to stick with the old one. So from now on I will be updating the good old wordpress blog.

I am now currently working at a company in Norway, Infotjenester AS as a manager of Application Management Department. This means I am in charge of the maintenance of all our products in the production environment. This means a lot of bugfixing, and continuous delivery strategies.

I am also in charge of the QA people here, who do their best to give our customer the greatest experience.

I guess I will be writing a little less about coding techniques, and more on CI, Devops and infrastructure

Winforms – Setting Focus() in Leave events

Although we live in 2012, there are still lots of code based on Winforms. And probably these codebases will exist in MANY years. But when I am set to do some UI stuff I struggle with the execution of events and what is allowed/disallowed.

Today I was working on a pretty simple task: When leaving a textbox, do some validation, and if the validation fails, set the focus to the textbox again. This is simple, but I had some stuff going on in the Enter event on the textbox and I didn’t want that to execute when the validation failed so I made this method

private void PreventKundenrLeave()
{
  tKundenr.Enter -= KundenrOnEnter;
  tKundenr.Focus();
  tKundenr.SelectAll();
  _oldValue = null;
  tKundenr.Enter += KundenrOnEnter;
}

This method ensures that the focus is still at the textbox if validation fails and and the Focus() method will be invoked AFTER the eventhandler on the Enter event has been removed.

But that didn’t work! The eventhandler still fired.

So I read in MSDN that you should absolutely under no circumstances do the following:

Do not attempt to set focus from within the Enter, GotFocus, Leave, LostFocus, Validating, or Validated event handlers.

The trick is to postpone the execution of the code until the Leave event is finished. This can be done with the BeginInvoke method.

 

 private void tKundenr_Leave(object sender, EventArgs e) {
        this.BeginInvoke((MethodInvoker)delegate { PreventKundenrLeave(); });
    }

Reading for knowledge

I rarely take the time to sit down and read a technical book. It’s sad, but I choose to deal with other things in life in addition to my work. To stay up to date technically I read my twitter feed every day (my favorites are @mossyblog @josefajardo and @kellabyte), and try to read some blog posts tweeted.

Some blogs are more interesting than others.Scott Hanselmann and Ayende Rahien is my favorites and I learn a lot from them. In fact, my next reading project is stepping a couple of years back in time, and read all the posts in Ayendes blog. There’s a lot of good discussions as well in these posts.

Presentation

February 3rd, I will hold my first presentation in a local school. 2 hours. I am very excited and also a bit nervous. It’s about some general information around software development and my experiences so far.

I will emphasize my experiences around an outsourcing project I was a part of a couple of years ago. How is it working with indians? How about differences in time, distance and culture? What did we learn?

2011 goals. And actual achievements

In the beginning of 2011 I wrote this blogpost about my goals for 2011. Here’s what I wrote:

Writing tests
…the right way and be convinced that, hey, this is actually very useful! I’ve worked with legacy code several years now, and it’s soo hard to start writing tests on existing stuff. Hope I can join a new project from scratch very soon .

Hold several presentations
…that’s right, I want to challenge myself to keep presentations for an audience (not just working colleagues). You learn so much from this and evolve in many ways. Maybe it will be for the norwegian .net user group?

Keep track on myself
…on everything I do. How productive am I? How many new technologies or frameworks have I learned? How many books have I read? How many training hours a week? Guess it’s my way to try to optimize my life and how to live it. Measure, measure, measure.

Finishing a marathon
…in under 200 minutes (3:20:00). Yup, I’m a running man and I’ve finished several marathons and my personal best is 3:28:47. In Berlin, september this year,  I will be running the worlds fastest marathon track and I hope for a new record also :)

Hard reality

And now for the truth:

Writing tests
I can honestly say that I haven’t wrote a single test in all of 2011. There’s no excuses, but all my work in 2011 has been on existing projects, legacy code. So the threshold for writing tests is a bit higher than when starting own projects from scratch

Hold several presentations
I’m afraid I haven’t held any presentations in 2011. BUT, I have booked a presentation on a school in february 2012.

Keep track on myself
I have analyzed some part of my activities in 2011:

Training: All my running laps are persistent
Music: All the best albums of 2011 are written down in a blogpost
Books: I read some books in January, but that’s it.
Movies: Sorry, I haven’t updated good movies since summer

Finishing a marathon
I finished on 3:24:40. Good result, but didn’t reach my 3:20:00 goal Trist fjes