Problems worthy of attack prove their worth by hitting back. —Piet Hein

Friday 8 November 2019

Bristech 2019

Yesterday I went to Bristech. What a great conference! It’s one day, three tracks.

The thing I like about it is the variety of the talks. It’s not organized around a single programming language or technology, so it’s easy to go to talks outside your usual sphere of interest (it’s quite hard not to). I went to talks about MLOps (keynote by Luke Marsden), the Language Server Protocol (by Krzysztof CieĊ›lak), a reactive JS compiler called Svelte (by Peter Allen), accessibility (by Svetlana Kouznetsova), Cloud Native ML (by Ant Kennedy), and building autonomous Mars rovers (by Mark Woods). I gave a talk on Single Cell data and algorithms (thanks to Steve Loughran for suggesting I submit it, as well as live tweeting it!).

Nic Hemley and the other organizers put a lot of work into lots of small details that made the day more memorable. To name a few: the Icelandic thunder clap; caterpillar and butterfly stickers to encourage attendees to talk to each other; lunchtime music and visuals; cinema popcorn. 

As a speaker I was very impressed by the care taken by track chairs on the speaker intros. Speakers were asked beforehand to fill in a short “interview” document, which chairs then used to write an intro for each speaker. My track chair was Hannah Smith who was also very good at gently reminding people in the Q&A session to ask questions rather than give their opinions. So we didn’t have any “this is more of a comment than a question” style ramblings, which I’m sure everyone was pleased about.

And the Watershed is a very nice venue, even if it gets “night club busy” between talks.

Great conference. Highly recommended.

Saturday 2 November 2019

How I manage my diabetes

I was diagnosed with Type 1 Diabetes (T1D) in February last year. Since then I have learnt a lot about the condition, and how to manage it. The first few months were quite turbulent, but now it’s become more settled and is a part of life, and thankfully I don’t spend every minute thinking about it. It never goes away though.

In this post I describe the technology I’m using to manage T1D. Every PWD (person with diabetes) is different, so what works for me won’t necessarily work for others. Also, some of these things don’t even work for me all the time, such is the unpredictable nature of diabetes. So there are definitely improvements I could make. I’ll mention some of them at the end of this piece.

Some of my diabetes kit

The main tools I use are a FreeStyle Libre to measure my blood glucose levels, and insulin pens to administer multiple daily injections (MDI) of insulin. I also use a number of apps and websites to manage the data.

Quite simply, the Libre is a superb piece of technology. It gives an amazing amount of insight into blood glucose levels - you can see what happens after you eat a particular food, or the effect of exercise, and even what happened to your levels during the night. I’ve been using one for over a year, and without it I really think I would be a lot more stressed about BG levels, and probably overcompensating for lows and oblivious to highs.

There are two parts to the Libre system: the sensor and the reader. The sensor is a small white disc that you stick to your upper arm. It has a small needle that sits under the skin where the glucose sensor is located. Each sensor lasts 14 days before it must be changed. The reader is a custom device, like a tiny phone (see picture above), but you can also use your phone to read the sensor using NFC. The sensor only stores 8 hours worth of measurements, so you need to tap the reader on the sensor at least that often to avoid gaps in history.

I use the reader rather than my phone to read the sensor (I only recently upgraded my phone to a version that can read the sensor). The reader can store 90 days of readings, so I periodically download the data as a backup and as a way to run different analyses on it.

Abbott, the company that manufactures the Libre, provides a desktop application to download the data from the Libre over USB. I wrote a script to upload the resulting data to a service called Nightscout. 


Nightscout is an open source project created by the #WeAreNotWaiting community to allow people with T1D to store their CGM data in the cloud. It was started as a way for parents of children with T1D to monitor their BG levels (particularly at night, without disturbing them), but it is now widely used by many in the T1D community, and forms a basis for DIY looping systems, like OpenAPS.

I use Nightscout as a historical data store, rather than for its realtime capabilities. I like Nightscout because it's open source and therefore not tied to a corporation (which could pull the service, change its terms, etc), but the downside is that you do have to run your own service, although that’s pretty easy to do on Heroku. 

Weekly BG summaries with dboard
Nightscout provides its own analytics, which are very useful. I also wrote a webpage to provide a weekly summary called dboard that reads data from Nightscout and shows a few key stats (time in range, average BG, estimated HbA1c) for the week, so I can see at a glance how the week was.

Turning to insulin, MDI uses two types of insulin - a basal dose of long-acting insulin (I use Levemir) every night before bed that provides a background supply of insulin over the day, and bolus doses of rapid-acting insulin (I use NovoRapid) taken before every meal or snack and which are adjusted to “cover” the carbs in that meal.

I don’t have an insulin pump. After I was diagnosed I assumed that I would eventually move on to one, since they offer a high degree of blood glucose control. However, my doctor said that since my control was very good using MDI, there wasn’t a strong reason to move to a pump. I'm still on MDI, and it works well for me. I was quite surprised to find that the needles on insulin pens are so fine that you can barely feel them, so injections are not normally painful. Extracting a drop of blood from a finger with a lancet for testing the blood glucose level hurts more.

Interestingly, it’s very likely that I’m still in the “honeymoon phase” of T1D, which is when the body still produces some insulin. This helps with control, and is probably another reason MDI works well for me. The literature on the duration of the honeymoon phase is maddeningly vague, but it’s typically between months and years. (One interesting paper I found suggests that doing regular exercise can prolong the honeymoon. A good reason to keep up the running…)

How do I know how much insulin to take? The nightly basal dose is fixed and the same every day - the dose was arrived at by trial and error soon after diagnosis, and it hasn’t changed much since.

Bolus doses for meals depend on two main variables: the amount of carbohydrate in the meal, and (to a lesser extent) the amount of exercise I’m expecting to do over the following few hours. This is the most fiddly part of managing T1D, since I have to work out the number of carbs in everything I eat. What’s more, I have to try to take my insulin at the right time before eating since injected insulin takes a while to get into the blood stream and become effective. To complicate things, some foods release glucose slowly, others a lot faster.

For carb counting at home where we cook most of our meals from scratch I wrote a web app called Ingreedy that calculates the number of carbs in a meal from its ingredients. This was especially useful in the early days when we had no idea about the carbs in particular foods. I also eat fewer carbs than I used to, around 120g a day. 

Carb counting a recipe with Ingreedy

I'm now better at eyeballing the (rough) number of carbs in food, which I need to do when eating out. If I’m unsure I use the Carbs & Cals app which has photos of foods for different portion sizes and their carb counts. 

After getting a carb count I have to convert this into the amount of insulin to take - there is a simple ratio to calculate this (e.g. 15g carbs translates to 1 unit of insulin), but the final amount is adjusted by my current BG reading, insulin already "on board" (i.e. in my body), and any exercise I’m going to do in the next few hours. I have a Google spreadsheet that acts as a food diary and does the calculation for me. Then I manually log the meal entry into Nightscout, so I have a log of number of carbs eaten and insulin taken. 

When I was first diagnosed I was told that it’s important not to always inject in the same place, since it can make injections less effective (and also cause damage to the area concerned). So I needed a scheme to rotate injection sites. At first I recorded a note of the site I just used on paper so I could choose the next one by looking at the note, but that quickly became tiresome, so I made a little gadget out of cardboard with a pin that I moved every time I did an injection. I still use that for my nightly basal injections.

For my meal bolus injections I now use a scheme that maps day of week and meal to an injection site on my tummy. This means I rotate through sites once a week, which is fine, and most importantly it’s stateless (i.e. I don’t need to remember where the previous site was). (Writing this prompts me to think about moving  to a similar scheme for my basal injections.)

Low blood sugar is an inevitable part of T1D. Sometimes you misjudge the amount of insulin and take too much (the pens I use can only administer whole units, so you have to round up or down to the nearest unit), or you do more exercise than you anticipated, or for some other reason your BG goes lower than you want it to. Although I can feel a hypo coming on, the Libre is a very useful tool for heading them off as I can see how quickly my BG is changing and possibly pre-empt a hypo. (For example, if my BG is a bit lower than expected and I’m going to go out soon I might eat something so I don’t go too low.)

When I’m having a hypo I will take glucose tablets, which I like for two reasons. First, they have a fixed dose (4g of carbs), and second they don’t taste that nice so you aren’t tempted to overdo the glucose and go too high.

I’ll also fall back to measuring BG with a finger prick test since blood generally gives a more accurate and more up-to-date reading than the Libre (which measures glucose from interstitial fluid, and has a 5 to 10 minute delay to changes in BG). I have a spare meter for this, or I use the Libre with a test strip. 

It’s easily overlooked, but making sure I have all the supplies I need requires a bit of organization. I thought about building an app that notifies me when I need to order something, but I never got round to it. Instead I have a calendar reminder each week to check supplies and order more of any that are running low. Low tech, but works for me so far.

Finally, there are a few things I’ve been thinking about changing or tweaking slightly.
  • Can I replace the Libre reader with the phone app?
  • Do I need dboard given that both Libre and Nightscout have good analytics?
  • Can I log insulin doses automatically (e.g. with CLIPSULIN)?
  • Do I need to log meals and carbs?
  • Is there a way of recording hypos? (Preferably automatically since when you are having a hypo you don’t think about logging stuff.)
  • How can we make it easier to move someone’s entire T1D data between systems? 

Friday 27 April 2018

Pastures new-ish

After nine and a half years, today is my last day at Cloudera. It’s difficult to write those words as so much of my life has been bound up with this company. On the day I started, I didn’t meet my co-workers as I was living several thousand miles away in a barn in Wales. (The others were in a borrowed meeting room in San Mateo.) As I leave I am still in a barn in Wales (different barn though), but a lot has happened in the intervening period.

On the personal side, my family and I lived in San Francisco during the early formative years of Cloudera, a time we will always treasure for the lifelong friendships we made.

On the professional side, it is no exaggeration to say that working at Cloudera has been the highlight of my career. I already knew that Hadoop was pretty special when I joined (I may have been biased as I was writing a book on it), but I had no idea how it would transform the industry and how it would be used in every sector you could imagine.

To all of you I have worked with over the last decade—at Apache, Cloudera and elsewhere, on many projects—I consider myself to be incredibly fortunate to have had the opportunity to work with you. Thank you.

So what’s next for me?

Jim Waldo, who worked on distributed systems at Sun, once said that he alternated six month periods between the lab and the outside world: in the lab he and his team built systems software, and in the outside world he saw how people used the system he was building. Doing so gave him valuable feedback on the system design, even though it was time away from being able to build the system.

In some ways this is another way of framing the explore/exploit tradeoff, where you decide between exploring new technological ground—building a new system—and exploiting that system to solve particular problems you are interested in, which is why you built the system in the first place. (Of course, this framing is oversimplified, since there are many people working on both parts simultaneously. It’s a useful way of thinking about things as an individual actor though.)

For the past few years I have been working on a few open source biology and healthcare projects (like GATK, Hail, and OHDSI). I think that the problems in biology are big enough and messy enough that new systems will need to be built. We can’t stop exploring the technological ground since the sheer amount of data will overwhelm even the best of today’s cutting-edge technology. (I like to cite the paper Big Data: Astronomical or Genomical? here for some concrete numbers.)

Having said that, there is still a lot of mileage left in our current crop of tools—which include Spark, TensorFlow, Jupyter, and the cloud. And this is what I am going to do: continue the work to apply tools like these to more bio projects, only now working as a freelancer. I plan to write more about what I’m up to on this blog, so please follow along.

Cloudera Inbox Zero for the first time ever!

Sunday 15 April 2018

Type 1 Diabetes

On 16 February this year I was diagnosed with Type 1 Diabetes.

I had been feeling under the weather - a bit weak, but also persistently thirsty and hungry. I would gulp down a large glass of water in one go with ease (a bit like I did when I was 10 years old after running around outside for hours on a hot day), and I would do this several times every day. I would eat a sandwich after dinner, even after having seconds. Strangely, I was losing weight despite eating a lot. And I found it hard to complete my usual morning run, and when I did manage it, it was noticeably slower than normal.

In retrospect these are all classic symptoms of type 1 diabetes: weight loss, increased thirst and hunger. My body was not producing enough insulin, which is needed to use the glucose in my blood. The weight loss occurred because my body was using fat reserves for energy. My heightened thirst was my body’s way of trying to flush the excess glucose from my system by getting me to urinate more.

So I went to see the doctor and she arranged a blood test, which I had the next day. That was at lunchtime on Friday, and the nurse who took my blood said it would be a week or two before the results came back. So I was surprised when the phone rang at 6pm, and the doctor’s receptionist asked me to come in. On Friday evening? Weren’t they closed then?

A few minutes later I went in, and she said that my blood glucose level was over 30, and that normally it would be 7. “Tom, you have diabetes.” I didn’t know what to say. I remember asking what the blood glucose level was measured in (millimoles per litre). (I’m always impressing on my daughters the importance of units in science.)

I also asked how they knew it was Type 1. Mainly from the symptoms - I'm quite skinny! Most people with diabetes have Type 2, which is characterised by insulin resistance: the body is still producing insulin, but it can’t use it as effectively. Less than 10% of diabetes sufferers have Type 1, and while most of those are children, adults can get the disease too.

The doctor sent me to the hospital straightaway so they could check if my body was coping with the high sugar levels. Left untreated, high blood glucose levels can lead to a dangerous condition called ketoacidosis.

I rang Eliane to tell her, and we both had a bit of a wobble. She drove over to take me to the hospital. We went to the Emergency Admissions Unit, where they would check my ketone levels (which, thankfully, were normal) and give me a shot of insulin. After that I could go home - there was no need to stay overnight, but they asked me to go back the next day and Sunday to be given insulin again.

The next day, Monday, I saw the Diabetic Specialist Nurse (DSN) who gave me a blood glucose monitor and insulin pens so I could manage the condition myself. She explained my new routine: before every meal and before bed I have to check my blood glucose level and inject insulin. The bedtime insulin is a slow-acting background insulin that lasts for almost a whole day, whereas the mealtime insulin is fast-acting and is meant to compensate for the blood glucose level rise caused by the carbohydrates in the meal. The idea is that you count the number of carbohydrates in the meal you are about to eat, then calculate the number of units of insulin that are needed to cover it.

It doesn’t sound like much, but I hadn’t really paid much attention to the nutritional composition of meals before. And nor had Eliane. We eat healthily, and mainly cook from scratch, but having to analyse each meal was a big change. In the first few weeks it felt like we were spending all our time analysing recipes. It gets easier, but it’s still time consuming.

The goal with diabetes is to keep the blood glucose level between 4 and 7 mmol/l. A person without diabetes has a pancreas that does this for them. Unfortunately, my pancreas has stopped performing this role, hence the need for injected insulin. There are two things to avoid: hyperglycaemia, which is when the blood glucose level is too high, and hypoglycaemia when it is too low.

Broadly speaking, hyperglycaemia has long term effects (such as cardiovascular problems), while hypoglycaemia needs to be treated immediately since its more severe form can require hospitalisation. Normally though, treating a mild “hypo” involves eating something with very fast acting sugar in it (like glucose tablets) and waiting 15 minutes for the level to get back in range. During this time you likely feel weak and shaky.

This chart shows all my blood glucose readings. In the first couple of weeks all my readings were out of range, but then they started stabilising, and now they are mainly in range.

When someone is newly diagnosed with diabetes in the UK, the full support network of the NHS swings into action. In addition to my GP, I have a diabetes consultant, two DSNs, and a dietitian. I saw all of these people in the first week, and I have ongoing support from the DSNs and the dietitian, who I can phone or email if I have a question, or need some help with my insulin dose adjustment. I have a recurring appointment with the consultant every six months. I’ve also been added to the system for annual screening for eye disease. The NHS also provides education programs for carb counting and insulin dose adjustment (one has the fabulous acronym DAFYDD - dose adjustment for your daily diet), as well as an excellent series of videos for newly diagnosed patients. All of this is free. And in Wales, where I live, there are no prescription charges for anyone (for people with diabetes in England, prescriptions are free too), so I don’t have to pay for the medical supplies that I now depend on every day. 

All of the medical staff that Eliane and I have encountered in the last two months have been unfailingly kind and supportive, even when working under pressure (like the first night in the EAU). It’s incredibly reassuring to have access to the resources the NHS provides. I would like to thank everyone there, along with my family, friends, and colleagues from work who have helped me get through the last two months. 

Wednesday 22 June 2016

Be Part of Something Bigger - Vote #Remain

David Cameron did not have to call this referendum. He did so in an attempt to settle the European issue within his own party, as there is no new EU treaty that we are voting on. The referendum is not binding either, as the legal blogger David Allen Green pointed out.

Far from quelling the debate within the Tory party, the lead up to the referendum has had the opposite effect. The debate over the last couple of months has been increasingly toxic, with both sides making outlandish claims. Parts of the Leave campaign have been xenophobic and racist, in an attempt to scare people to leaving the EU - this is the true Project Fear. And then last week the appalling murder of the Labour MP Jo Cox brought about some reflection on how we’ve moved away from a more respectful, kinder politics. In the words of Stephen Kinnock, "When insecurity, fear and anger are used to light a fuse, an explosion is inevitable.”

But there is a referendum tomorrow, so we have a duty to vote. The vote is about a host of issues, and on all of them I believe we are better staying as a member of the EU. In my mind it boils down to being a part of something bigger than yourself. This is true on a personal level - being part of a company, a team, a club, or an organisation allows you to achieve more than if you go it alone. I’ve seen this in my professional life where loose-knit groups of programmers build open source software that an individual could never dream of. Of course, where there are many personalities pulling in different directions you get conflict, things get messy, compromises are needed, and you don’t always get your own way. But on some decisions you do have influence, and you do get to shape the results.

Being a part of the EU is about the UK being a part of something bigger, and being able to influence policy on issues that affect the UK. The world is a messy chaotic place, and there are many deeply-ingrained, complex problems that require complex policy interventions. Climate change, migration, tax havens, peace - to name a few - all of these need a coordinated approach that cross national borders. Leaving would squander our influence in attacking these problems, while doing nothing to solve them - for us or for the the rest of the world.

One of the more worrying themes of the Leave campaign is not to trust the experts. This allows them to conveniently dismiss the overwhelming opinion amongst economists that Brexit would mean the UK is worse off outside the EU. It’s like climate change denial, and running a country with that kind of gut-feeling policy making is terrifying.

Britain has been at its best when it has been an outward-looking nation, one that works with others and trades with others. That’s why I am going to vote to Remain in the EU.