Daily Archives

One Article

Posted by knomee on

Knomee Sense-making Algorithm is Grown from your Data

In his famous 1995 book “Out of Control”, Kevin Kelly wrote that smart systems should be “grown, not designed”. He meant that intelligent behavior should emerge from collected data and experience and not engineered in a top down way.

This is exactly how Knomee “smarts” (as in “self-tracking with sense”) have been developed: the pattern-detection algorithm is grown not designed. Knomee holds an “algorithm factory” in your smartphone, that “grows” a specific algorithm from your data, which is unique by construction.

 

(1) Knomee uses AI for Forecast and Statistical Validation

Knomee uses a number of techniques to provide insights and feedback. Most of it is classical statistical lore, but Knomee uses artificial intelligence to craft an algorithm that tries to « understand » your data, which means here to detect a collection of relevant patterns. This algorithm then serves two purposes. First it is use as a « forecasting » oracle. This is useful since it means that when you open Knomee the tracker sliders are usually positioned pretty close to where you would like them to be (10% to 15% error on average). This makes tracking faster ... and fun. This is the most convincing usage of « forecasts »: there is no way that Knomee could predict your future with the small amount of data that you track, but making Knomee « active » makes it faster to use ... and more fun ! Once you have enough data, it is actually amusing to see when Knomee gets it right and when it does not (usually, these are the most interesting self-tracking moments). The second use of this « smart » algorithm is to evaluate the relevance of more classical statistical observation. The scoring that Knomee reports about the influence of factors (tracker, time, location) is a combination of correlation and contribution to the AI insights.

 

(2) How to Grow a Unique Algorithm from your Data 

The emphasis in Knomee is on robustness much more than on precision. In the world of “small time series” (which is precisely why you get with bio-rhythms), high fidelity forecasting is an illusion and the common curse is “overfitting”: trying desperately to see some sense where there is none.

This forecasting algorithm is produced using program synthesis and reinforcement learning. Knomee has crafted an abstract description of meaningful patterns for biorhythm time series (a term algebra) and use randomization techniques to explore the wide space of possible variations. It then selects an evolutionary meta-search method to optimize the programs that better fit (reinforcement) according to their ability to explain the data. The search space includes the set of classical techniques such as k-neighbors or regression, but the evolutionary control protocol is geared at escaping the classical overfitting trap (after all, we never expect you to self-track a large amount of data).

We call the meta-algorithm that runs in your smartphone RIES for Randomized Incremental Evolutionary Search - it is a short-time series variation of techniques that were developed many years ago.  It is part of a method named EMLA (Evolutionary Machine Learning Agents); the « Incremental » specificity of the Knomee implementation is that it is optimized to fit the limited capacity of a smartphone (from a machine learning perspective).

 

(3) This Algorithm is Unique to You because You are Unique 

The RIES "algorithm factory" produces an algorithm that is truly unique because it is grown from your data. This algorithm is born on your phone and stays there. No-one will have access to the set of insights that is embedded into this algorithm. This approach is not meant for scaling or abstracting from multiple individuals.

The most interesting characteristic of EMLA is its ability to avoid false positives and let you know if your data has no relevant or statistically significant insights. This is especially critical for users because we get many of our quests wrong! We believe that we could improve some aspect of our well-being by changing our behavior ... and it simply does not work. As Mark Twain famously quoted « It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so. »

If you track “noise” (random data), Knomee will avoid overfitting and tell you that nothing much can be learned from your self-tracking data. It may does not sound like much, but it is a great feature of Knomee and something that distinguishes it from dubious so-called machine learning applications.