The Ikawa Sample Roaster & survivor bias

One of the many routine tasks in a coffee roasting business is sample roasting.  We’re constantly receiving small quantities of different coffees from green coffee importers, and we need to figure out whether it makes sense to buy any of them.  Sample roasting is time consuming, and tbh, not a lot of fun.  The goal is to roast the different coffees to a similar roast/development level so that they can be evaluated apples-to-apples (without differences imparted by variations in roast).  So the goal isn’t necessarily to bring about the best in each coffee, but just to stick to a process. It can also be stressful: typically you roast a lot of samples at once, and some green coffee importers only send enough of a sample to allow for one, possibly two roasts….you don’t want to make a mistake as green coffee buying is a speed game, coveted coffees go extremely quickly.

Enter the Ikawa. It looks like a glorified electric air popper, and it kind of is. However, it is a lot more robust, has a fast thermocouple (which reads the air temp right above the rotating fluid-bed-y coffee mass as a proxy for bean temp) and a PID linked to an iOS app which allows you to program profiles for the time-temp path you’d like the temperature readings to travel. It also has a very clever built in chaff separation mechanism, and a roasted bean cooling function. Finally, it has created a great community around the sharing of roast profiles, and it is easy to send profiles to others so that you can roughly sample roast in the same way.

I could write another blog post on profile creation and the different approaches for different origins/processing methods, and whether I think a PID which forces the roast along a prescribed path is an optimal way to go about roasting coffee.  But suffice to say that basically, the Ikawa dramatically simplifies workflow and churns out pretty decent sample roasts. Load a profile onto the machine via a Bluetooth enabled iOS device, hit a button, wait until the Ikawa warms to the charge temp, load the green coffee, and wait for a chime to go off 7-8 minutes later to let you know the roast is done. You can even do other things while the coffee is roasting which is kind of incredible for a roaster.

So it is no wonder that the Ikawa roaster is taking the roasting and green coffee end of the supply chain by storm. It makes sample roasting so much easier and efficient. In tandem with this, I see more green coffee importers offering roasted samples of coffees in lieu of green coffee samples. Makes sense: it’s less work for the roaster, the Importer is ensured that the roaster is giving the coffee a fair shake (ie hasn’t done horrible things to the coffee while sample roasting, and is actually going to cup the samples!) and reduces waste (just one 50g sample bag, not a larger amount of green, some of which will inevitably be wasted).

When you receive roasted coffee samples, however, you are deprived the opportunity to examine and grade the green. You can’t count defect number and type, nor can you read the moisture and the like. So you need to be sure that the roasted coffee you receive accurately reflects what is inside the 150lb bags of green coffee you’ll be buying. Every green importer I’ve dealt with has been honest to a fault and would never groom green samples and remove defects. However, when it comes to roasted samples, I think there is an inadvertent source of bias that perhaps many importers and roasters don’t consider when evaluating the coffee.

The Ikawa sample roaster is pretty close to being a purely convective roaster, the coffee makes very little if any contact with metal surfaces, and thus heat transfer occurs entirely via hot air.  The green coffee is held aloft by the force of the air, kind of in the same way that a hovercraft floats over the water or ground. So airflow is by necessity high, and I believe proportionally higher in the Ikawa than in more traditional open faced sample roasters (or any traditional drum roaster for that matter). Ikawa cleverly uses this high airflow to loft away the chaff (which is far less dense than the coffee) into a catch “cup”. But quakers and many other types of defective beans are also less dense than the more perfect ones, and these are jettisoned with an alarmingly high efficiency into the same catch cup too. So, if you are in a hurry and dump the contents of the catch cup without pulling the quakers (and at times it can be hard to see the light colored beans in the fluffy chaff), you are basically grooming the coffee of defects and creating a sample that is better than – and not accurately representative of – the actual coffee lot under consideration. It is certainly the case that all roasters will suck up some quakers into their chaff-clearing cyclones; but from what I’ve seen, the Ikawa removes more defects as a percentage of the total defect pool than other roasters.

This might not make too much of a difference for extremely clean and well processed coffees with very few defects/quakers (typically these are very high scoring and pricey coffees). But in most samples of even decent specialty coffee, you’ll find a surprising number of quakers and other mild defects. In mid-range specialty coffee, I often find around half the quakers and other obvious defects in the catch cup; in a 50g Ikawa roast, this means that I might find as many as 3-5 defective beans in the cup (on the order of 1% of the total 50g!)  To me, this is pretty significant and to the point where I believe it’s impacting the cupping scores. And also quite telling is that most of the beans that end up in the catch cup are quakers and not viable coffee (good beans also make it into the catch cup, but in lower numbers, and thus at a much lower percentage of the total amount of good beans in the sample).

This is not to disparage the Ikawa, it is an awesome machine. I just want to point out that a by-product of its chaff clearing system – that is ironically perhaps too good – is that the user could be left with a cleaned up pool of coffee and should factor that into the evaluation process.

2 Replies to “The Ikawa Sample Roaster & survivor bias”

  1. awesome post- i agree the ikawa is great tool and saves a lot of time in the lab. Another interesting thing to consider is how an almost all convection roaster would compare to its drum relatives. Though all convection can certainly produce great coffee no doubt, it still produces a slightly different end result as far as taste goes. How big the difference is varies but say you are production roasting on a ug22 I feel id be more apt to want to use a traditional sample drum for consistency and some insight on how it will do in your particular roaster. But for the price, the ikawa is almost a no brainer especially for traveling.

Leave a Reply

Your email address will not be published. Required fields are marked *