Jekyll2018-05-02T00:04:02+00:00http://guidomb.blog/exit0experiences & thoughts about software engineering and entrepreneurship.Advanced code generation2018-04-23T03:00:00+00:002018-04-23T03:00:00+00:00http://guidomb.blog/swift/2018/04/23/advanced-code-generation<p>In the previous post, <a href="/swift/2018/04/09/how-to-represent-code-invariants.html">“How to represent code invariants”</a>, I described how code generation and meta-programming can help us represent code invariants. Although the proposed solution works there was a small bug. Let’s take a look at the template I used to generate the extension for the type that conformed to <code class="highlighter-rouge">AutoRangeMappable</code> which added a computed property to list the value of all properties of type <code class="highlighter-rouge">SpreadSheetRange</code>.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{% for type in types.implementing.AutoRangeMappable %}
extension {{ type.name }} {
static let rangesCount = {{ type.instanceVariables.count }}
public var ranges: [SpreadSheetRange] {
return [
{% for variable in type.instanceVariables %}{% if variable.type.name == "SpreadSheetRange" %} {{ variable.name }}{% if not forloop.last %},{% endif %}{% endif %}
{% endfor %}
]
}
}
{% endfor %}
</code></pre></div></div>
<p>As you can see, when defining the computed property <code class="highlighter-rouge">ranges</code>, the template iterates over all instance variables and performs a check to see wether the instance variable’s type is <code class="highlighter-rouge">SpreadSheetRange</code>. If that check returns <code class="highlighter-rouge">true</code> it adds the instance variable to the returning array, otherwise the instance variable is ignored.</p>
<p>Did you spot the bug in the template? The problem is on how the <code class="highlighter-rouge">rangesCount</code> variable is defined. The template returns the total amount of instance variables, without taking into account that not all instance variables may be of type <code class="highlighter-rouge">SpreadSheetRange</code>.</p>
<p>The problem is that <a href="https://github.com/kylef/Stencil">Stencil templates</a> don’t have a way of executing “complex” Swift expressions. Taken from <a href="http://stencil.fuller.li/en/latest/templates.html#variables">Stencil’s documentation</a>:</p>
<blockquote>
<p>Stencil will look up the variable inside the current variable context andevaluate it. When a variable contains a dot, it will try doing the following lookup:</p>
<ul>
<li>Context lookup</li>
<li>Dictionary lookup</li>
<li>Array lookup (first, last, count, index)</li>
<li>Key value coding lookup</li>
<li>Type introspection</li>
</ul>
</blockquote>
<p><code class="highlighter-rouge">type.instanceVariables.count</code> works because <code class="highlighter-rouge">type.instanceVariable</code> returns an array by doing a key value coding lookup (or type introspection, I’m not really sure which one) and then <code class="highlighter-rouge">.count</code> works because it’s one of the supported array lookup operations. Ideally I’d like to do something like <code class="highlighter-rouge">type.instanceVariables.filter({ $0.type.name == "SpreadSheetRange" }).count</code> but that won’t work because of how variable evaluation works in Stencil.</p>
<p>So what can you do in this scenarios? Well, thankfully Sourcery provides another templating language (which is not as documented as Stencil is) which is Swift template. This template is way more powerful because it let us execute any piece of Swift code we want. All you need to do to use a Swift template is change the extension of the file where you declared you template from <code class="highlighter-rouge">.stencil</code> to <code class="highlighter-rouge">.swifttemplate</code>. Also the syntax changes a little bit. Going back to the original problem of getting the amount of instance variables of type <code class="highlighter-rouge">SpreadSheetRange</code>, using Swift template we can define the following template:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><% for type in types.all where type.implements["AutoRangeMappable"] != nil { %>
extension <%= type.name %> {
static let rangesCount = <%= type.instanceVariables.filter({ $0.type?.name == "SpreadSheetRange" }).count %>
var ranges: [SpreadSheetRange] {
return [
<% for instanceVariable in type.instanceVariables where instanceVariable.type?.name == "SpreadSheetRange" { -%>
<%= instanceVariable.name %>,
<% } %>
]
}
}
<% } %>
</code></pre></div></div>
<p>As you can see the overall structure is pretty similar. Important things to note:</p>
<ul>
<li>Instead of <code class="highlighter-rouge">{% %}</code> to execute control flow statements you need to use <code class="highlighter-rouge"><% %></code>.</li>
<li>Anything inside <code class="highlighter-rouge"><% %></code> needs to be valid Swift syntax, otherwise you’ll get a compiler error.</li>
<li>Instead of <code class="highlighter-rouge">{{ }}</code> to evaluate a variable you need to use <code class="highlighter-rouge"><%= %></code> which will print the result of evaluating the given Swift expression.</li>
<li>You loose some of the syntax sugar like <code class="highlighter-rouge">types.implementing.AutoRangeMappable</code> which in Swift template translates to <code class="highlighter-rouge">for type in types.all where type.implements["AutoRangeMappable"] != nil</code>. Here we are using Swift’s <code class="highlighter-rouge">for-in</code> statement combined with a <code class="highlighter-rouge">where</code> clause to filter out elements that don’t satisfy the given condition. Sourcery’s Swift API to access all confirming types is provided via the <code class="highlighter-rouge">implements</code> property that returns a <code class="highlighter-rouge">[String: Type]</code> (check the documentation for Sourcery’s Swift API <a href="https://cdn.rawgit.com/krzysztofzablocki/Sourcery/master/docs/Classes/Type.html#/c:@M@SourceryRuntime@objc(cs)Type(py)implements">here</a>).</li>
<li>For some reason when using <code class="highlighter-rouge"><% %></code> or <code class="highlighter-rouge"><%= %></code>the generated code will have extra new lines where the Swift statement or expression was executed. You can avoid extra new lines by using <code class="highlighter-rouge">-</code>, like <code class="highlighter-rouge"><% -%></code> or <code class="highlighter-rouge"><%= -%></code> but indentation is not properly done (maybe a bug with swift template).</li>
</ul>
<p>We are now able to properly generate the <code class="highlighter-rouge">rangesCount</code> property by filtering out all instance variable which type is not <code class="highlighter-rouge">SpreadSheetRange</code> and then asking the <code class="highlighter-rouge">count</code> to the resulting array.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>static let rangesCount = <%= type.instanceVariables.filter({ $0.type?.name == "SpreadSheetRange" }).count %>
</code></pre></div></div>
<p>The <code class="highlighter-rouge">ranges</code> computed property is generated in a similar way as in the Stencil template adapting the syntax to the Swift template one.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>var ranges: [SpreadSheetRange] {
return [
<% for instanceVariable in type.instanceVariables where instanceVariable.type?.name == "SpreadSheetRange" { -%>
<%= instanceVariable.name %>,
<% } %>
]
}
</code></pre></div></div>
<p>Overall Swift templates provide a more flexible experience at the expense of longer compile times and a harder debugging experience (Sourcery just prints raw compiler errors to the console).</p>
<p>In the next blog post we will learn how to make this template more generic in order to be able to reuse it to list other types of instance variables instead of just <code class="highlighter-rouge">SpreadSheetRange</code> variables.</p>In the previous post, “How to represent code invariants”, I described how code generation and meta-programming can help us represent code invariants. Although the proposed solution works there was a small bug. Let’s take a look at the template I used to generate the extension for the type that conformed to AutoRangeMappable which added a computed property to list the value of all properties of type SpreadSheetRange.How to represent code invariants2018-04-09T12:00:00+00:002018-04-09T12:00:00+00:00http://guidomb.blog/swift/2018/04/09/how-to-represent-code-invariants<p>I wanted to write a bit about how we represent code invariants, specifically when those invariants cannot be easily expressed or enforced by the language, in this case the Swift programming language. When we talk about invariants <a href="https://en.wikipedia.org/wiki/Invariant_(computer_science)">Wikipedia’s definition</a> explains it quite well:</p>
<blockquote>
<p>In <a href="https://en.wikipedia.org/wiki/Computer_science">computer science</a>, an <strong>invariant</strong> is a condition that can be relied upon to be true during execution of a program, or during some portion of it. It is a <a href="https://en.wikipedia.org/wiki/Logical_assertion">logical assertion</a> that is held to always be true during a certain phase of execution. For example, a <a href="https://en.wikipedia.org/wiki/Loop_invariant">loop invariant</a> is a condition that is true at the beginning and end of every execution of a loop.</p>
</blockquote>
<p>Basically something that we know it’s always true and won’t change. Otherwise somebody made a serious mistake and our program would probably crash. Now that we have defined some terminology lets jump right down to the code.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">struct</span> <span class="kt">RangeMapper</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">title</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="k">let</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="k">let</span> <span class="nv">attributes</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="kd">public</span> <span class="nf">init</span><span class="p">(</span><span class="nv">title</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">,</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">,</span> <span class="nv">attributes</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">)</span> <span class="p">{</span>
<span class="k">self</span><span class="o">.</span><span class="n">title</span> <span class="o">=</span> <span class="n">title</span>
<span class="k">self</span><span class="o">.</span><span class="n">description</span> <span class="o">=</span> <span class="n">description</span>
<span class="k">self</span><span class="o">.</span><span class="n">attributes</span> <span class="o">=</span> <span class="n">attributes</span>
<span class="p">}</span>
<span class="k">var</span> <span class="nv">ranges</span><span class="p">:</span> <span class="p">[</span><span class="kt">SpreadSheetRange</span><span class="p">]</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[</span><span class="n">title</span><span class="p">,</span> <span class="n">description</span><span class="p">,</span> <span class="n">attributes</span><span class="p">]</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="highlighter-rouge">RangeMapper</code> is a type that represents a set of <code class="highlighter-rouge">SpreadSheetRange</code>s that have a particular semantic associated with it. A <code class="highlighter-rouge">SpreadSheetRange</code> is a simple type that basically wraps a <code class="highlighter-rouge">String</code> and when created it verifies that the given string follows certain rules. In particular it should be a valid cells range identifier expressed in the <a href="https://developers.google.com/sheets/api/guides/concepts#a1_notation">A1 notation</a>. This notation is the one you may use to refer to a range of cells in a Google spread sheet, e.g: <code class="highlighter-rouge">Sheet1!A1:D4</code>. The following is a possible implementation of that type without the initializer’s implementation because it is a little bit cumbersome:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">enum</span> <span class="kt">SpreadSheetRange</span><span class="p">:</span> <span class="kt">CustomStringConvertible</span><span class="p">,</span> <span class="kt">Equatable</span> <span class="p">{</span>
<span class="kd">public</span> <span class="kd">enum</span> <span class="kt">CellRange</span><span class="p">:</span> <span class="kt">CustomStringConvertible</span><span class="p">,</span> <span class="kt">Equatable</span> <span class="p">{</span>
<span class="k">case</span> <span class="nf">allCellsInRows</span><span class="p">(</span><span class="nv">fromRow</span><span class="p">:</span> <span class="kt">UInt</span><span class="p">,</span> <span class="nv">toRow</span><span class="p">:</span> <span class="kt">UInt</span><span class="p">)</span>
<span class="k">case</span> <span class="nf">allCellsInColumn</span><span class="p">(</span><span class="nv">columnName</span><span class="p">:</span> <span class="kt">String</span><span class="p">)</span>
<span class="k">case</span> <span class="nf">allCellsInColumnFromRow</span><span class="p">(</span><span class="nv">columnName</span><span class="p">:</span> <span class="kt">String</span><span class="p">,</span> <span class="nv">fromRow</span><span class="p">:</span> <span class="kt">UInt</span><span class="p">)</span>
<span class="k">case</span> <span class="nf">range</span><span class="p">(</span><span class="nv">from</span><span class="p">:</span> <span class="kt">String</span><span class="p">,</span> <span class="nv">to</span><span class="p">:</span> <span class="kt">String</span><span class="p">)</span>
<span class="kd">public</span> <span class="k">var</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">String</span> <span class="p">{</span>
<span class="k">switch</span> <span class="k">self</span> <span class="p">{</span>
<span class="k">case</span> <span class="o">.</span><span class="nf">allCellsInRows</span><span class="p">(</span><span class="k">let</span> <span class="nv">fromRow</span><span class="p">,</span> <span class="k">let</span> <span class="nv">toRow</span><span class="p">):</span>
<span class="k">return</span> <span class="s">"</span><span class="se">\(</span><span class="n">fromRow</span><span class="se">)</span><span class="s">:</span><span class="se">\(</span><span class="n">toRow</span><span class="se">)</span><span class="s">"</span>
<span class="k">case</span> <span class="o">.</span><span class="nf">allCellsInColumn</span><span class="p">(</span><span class="k">let</span> <span class="nv">columnName</span><span class="p">):</span>
<span class="k">return</span> <span class="s">"</span><span class="se">\(</span><span class="n">columnName</span><span class="se">)</span><span class="s">:</span><span class="se">\(</span><span class="n">columnName</span><span class="se">)</span><span class="s">"</span>
<span class="k">case</span> <span class="o">.</span><span class="nf">allCellsInColumnFromRow</span><span class="p">(</span><span class="k">let</span> <span class="nv">columnName</span><span class="p">,</span> <span class="k">let</span> <span class="nv">fromRow</span><span class="p">):</span>
<span class="k">return</span> <span class="s">"</span><span class="se">\(</span><span class="n">columnName</span><span class="se">)\(</span><span class="n">fromRow</span><span class="se">)</span><span class="s">:</span><span class="se">\(</span><span class="n">columnName</span><span class="se">)</span><span class="s">"</span>
<span class="k">case</span> <span class="o">.</span><span class="nf">range</span><span class="p">(</span><span class="k">let</span> <span class="nv">from</span><span class="p">,</span> <span class="k">let</span> <span class="nv">to</span><span class="p">):</span>
<span class="k">return</span> <span class="s">"</span><span class="se">\(</span><span class="n">from</span><span class="se">)</span><span class="s">:</span><span class="se">\(</span><span class="n">to</span><span class="se">)</span><span class="s">"</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nf">init</span><span class="p">?(</span><span class="n">from</span> <span class="nv">string</span><span class="p">:</span> <span class="kt">String</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Somewhat long implementation that verifies all possible valid</span>
<span class="c1">// A1 notation cases.</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">case</span> <span class="nf">allCellsInSheet</span><span class="p">(</span><span class="nv">sheetName</span><span class="p">:</span> <span class="kt">String</span><span class="p">)</span>
<span class="k">case</span> <span class="nf">cellRange</span><span class="p">(</span><span class="nv">sheetName</span><span class="p">:</span> <span class="kt">String</span><span class="p">?,</span> <span class="nv">cellRange</span><span class="p">:</span> <span class="kt">CellRange</span><span class="p">)</span>
<span class="kd">public</span> <span class="k">var</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">String</span> <span class="p">{</span>
<span class="k">switch</span> <span class="k">self</span> <span class="p">{</span>
<span class="k">case</span> <span class="o">.</span><span class="nf">allCellsInSheet</span><span class="p">(</span><span class="k">let</span> <span class="nv">sheetName</span><span class="p">):</span>
<span class="k">return</span> <span class="n">sheetName</span>
<span class="k">case</span> <span class="o">.</span><span class="nf">cellRange</span><span class="p">(</span><span class="k">let</span> <span class="nv">sheetName</span><span class="p">?,</span> <span class="k">let</span> <span class="nv">cellRange</span><span class="p">):</span>
<span class="k">return</span> <span class="s">"</span><span class="se">\(</span><span class="n">sheetName</span><span class="se">)</span><span class="s">!</span><span class="se">\(</span><span class="n">cellRange</span><span class="se">)</span><span class="s">"</span>
<span class="k">case</span> <span class="o">.</span><span class="nf">cellRange</span><span class="p">(</span><span class="o">.</span><span class="k">none</span><span class="p">,</span> <span class="k">let</span> <span class="nv">cellRange</span><span class="p">):</span>
<span class="k">return</span> <span class="n">cellRange</span><span class="o">.</span><span class="n">description</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">public</span> <span class="nf">init</span><span class="p">?(</span><span class="n">from</span> <span class="nv">string</span><span class="p">:</span> <span class="kt">String</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Implementation</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The reason I’m talking about spread sheets and cell ranges is because I’m writing a program that needs to extract data from several Google spread sheets and do some magic with it. In one of those spread sheets there is structured data and that data could be represented with a struct like the following one:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">struct</span> <span class="kt">Ability</span> <span class="p">{</span>
<span class="kd">struct</span> <span class="kt">Attribute</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">name</span><span class="p">:</span> <span class="kt">String</span>
<span class="k">let</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">String</span>
<span class="k">let</span> <span class="nv">value</span><span class="p">:</span> <span class="kt">Int</span>
<span class="p">}</span>
<span class="k">let</span> <span class="nv">title</span><span class="p">:</span> <span class="kt">String</span>
<span class="k">let</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">String</span>
<span class="k">let</span> <span class="nv">attributes</span><span class="p">:</span> <span class="p">[</span><span class="kt">Attribute</span><span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This structured data is spread throw out several spread sheets so I need to fetch from all of this spread sheets the cell ranges that contain this structured data. The spread sheet API has a method called <a href="https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets.values/batchGet">batchGet</a> to fetch in one API call several cell ranges. It is important to note that a particular cell range could be translated to several <code class="highlighter-rouge">Ability</code> properties, meaning that there is not one to one relation between cell ranges and <code class="highlighter-rouge">Ability</code> properties.</p>
<p>That is why the <code class="highlighter-rouge">RangeMapper</code> type exists, to define the cell range for each <code class="highlighter-rouge">Ability</code>’s property. We can then define an array of <code class="highlighter-rouge">RangeMapper</code> that need to be fetched and translate that to a list of ranges to pass to the <code class="highlighter-rouge">batchGet</code> method.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">mappers</span><span class="p">:</span> <span class="p">[</span><span class="kt">RangeMapper</span><span class="p">]</span> <span class="o">=</span> <span class="c1">//....</span>
<span class="k">let</span> <span class="nv">cellRanges</span><span class="p">:[</span><span class="kt">SpreadSheetRange</span><span class="p">]</span> <span class="o">=</span> <span class="n">mappers</span><span class="o">.</span><span class="n">flatMap</span> <span class="p">{</span> <span class="nv">$0</span><span class="o">.</span><span class="n">ranges</span> <span class="p">}</span>
</code></pre></div></div>
<p>It is out of the scope of this blog post to explain how the fetching and parsing logic works but believe than inside the code that parses the response from <code class="highlighter-rouge">batchGet</code> in order to get <code class="highlighter-rouge">[Ability]</code> as a result I needed to reference the amount of <code class="highlighter-rouge">SpreadSheetRange</code>s needed to parse an <code class="highlighter-rouge">Ability</code> value.</p>
<p>This is the invariant that I know holds true as long as I need 3 ranges to extract the data needed to create an <code class="highlighter-rouge">Ability</code> value. I could add a comment explaining what’s the reason behind the magic number 3. I could then extract it into a constant and forget about it but this is the sort of trap I’ve fallen down so many times before. What happens if the requirements change? Lets say that somebody decides to add new data and I need more ranges to extract it in order to successfully parse an <code class="highlighter-rouge">Ability</code> value, then I would need to update that magic number, which if extracted into a constant would only need to update it one place and if I have proper unit tests any issue should arise. The truth is that real world software development is a messy process and some of this measures could fail or not be implemented at all. Also I do have an OCD with things that I know could be automated, so this gave an excuse to think about how to reflect invariants that cannot be easily expressed with the tools we have and can evolve naturally as other parts of the system changes.</p>
<h3 id="option-1-use-a-constant">Option 1: Use a constant</h3>
<p>The first options is the simplest one, which is adding the <code class="highlighter-rouge">rangesCount</code> static property. This has the disadvantage of remembering to keep in sync the <code class="highlighter-rouge">rangesCount</code> static property with the <code class="highlighter-rouge">ranges</code> computed property. We also need to remember to keep in sync the actual instance variables of type <code class="highlighter-rouge">SpreadSheetRange</code> with ranges array. Both issues could be mitigated by having unit tests.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">struct</span> <span class="kt">RangeMapper</span> <span class="p">{</span>
<span class="kd">static</span> <span class="kd">func</span> <span class="k">let</span> <span class="nv">rangesCount</span> <span class="o">=</span> <span class="mi">3</span>
<span class="k">let</span> <span class="nv">title</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="k">let</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="k">let</span> <span class="nv">attributes</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="kd">public</span> <span class="nf">init</span><span class="p">(</span><span class="nv">title</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">,</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">,</span> <span class="nv">attributes</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">)</span> <span class="p">{</span>
<span class="k">self</span><span class="o">.</span><span class="n">title</span> <span class="o">=</span> <span class="n">title</span>
<span class="k">self</span><span class="o">.</span><span class="n">description</span> <span class="o">=</span> <span class="n">description</span>
<span class="k">self</span><span class="o">.</span><span class="n">attributes</span> <span class="o">=</span> <span class="n">attributes</span>
<span class="p">}</span>
<span class="k">var</span> <span class="nv">ranges</span><span class="p">:</span> <span class="p">[</span><span class="kt">SpreadSheetRange</span><span class="p">]</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[</span><span class="n">title</span><span class="p">,</span> <span class="n">description</span><span class="p">,</span> <span class="n">attributes</span><span class="p">]</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="option-2-use-key-paths">Option 2: Use key paths</h3>
<p>The second option is to only have one place where the actual ranges are assembles, in this case in a static property called <code class="highlighter-rouge">ranges</code>. Because this is a static property we need to use <code class="highlighter-rouge">KeyPath</code>s if we want to have an array to which we can ask the amount of elements it has. In this case the only think we need to keep in sync is the list of key path with the instance variables of type <code class="highlighter-rouge">SpreadSheetRange</code>. I consider option 2 to be superior to option 1 because it reduces the amount of things to keep in sync without adding too much complexity.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">struct</span> <span class="kt">RangeMapper</span> <span class="p">{</span>
<span class="kd">static</span> <span class="k">var</span> <span class="nv">ranges</span><span class="p">:</span> <span class="p">[</span><span class="kt">KeyPath</span><span class="o"><</span><span class="kt">RangeMapper</span><span class="p">,</span> <span class="kt">SpreadSheetRange</span><span class="o">></span><span class="p">]</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[</span>
<span class="err">\</span><span class="kt">RangeMapper</span><span class="o">.</span><span class="n">title</span><span class="p">,</span>
<span class="err">\</span><span class="kt">RangeMapper</span><span class="o">.</span><span class="n">description</span><span class="p">,</span>
<span class="err">\</span><span class="kt">RangeMapper</span><span class="o">.</span><span class="n">attributes</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="k">let</span> <span class="nv">title</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="k">let</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="k">let</span> <span class="nv">attributes</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="kd">public</span> <span class="nf">init</span><span class="p">(</span><span class="nv">title</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">,</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">,</span> <span class="nv">attributes</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">)</span> <span class="p">{</span>
<span class="k">self</span><span class="o">.</span><span class="n">title</span> <span class="o">=</span> <span class="n">title</span>
<span class="k">self</span><span class="o">.</span><span class="n">description</span> <span class="o">=</span> <span class="n">description</span>
<span class="k">self</span><span class="o">.</span><span class="n">attributes</span> <span class="o">=</span> <span class="n">attributes</span>
<span class="p">}</span>
<span class="k">var</span> <span class="nv">ranges</span><span class="p">:</span> <span class="p">[</span><span class="kt">SpreadSheetRange</span><span class="p">]</span> <span class="p">{</span>
<span class="k">return</span> <span class="kt">RangeMapper</span><span class="o">.</span><span class="n">ranges</span><span class="o">.</span><span class="n">map</span> <span class="p">{</span> <span class="k">self</span><span class="p">[</span><span class="nv">keyPath</span><span class="p">:</span> <span class="nv">$0</span><span class="p">]</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="option-3-use-sourcery">Option 3: Use Sourcery</h3>
<p>The third option is to use <a href="https://github.com/krzysztofzablocki/Sourcery">Sourcery</a> to automatically generate all the code to avoid having to keep state in sync. I consider this to be the superior solution to all previous options at the expense of having to introduce a new tool as part of the build process.</p>
<p>For those who don’t know what Sourcery is, basically allows you to generate code based on templates that can introspect your source code. In this case we are defining an extension for every type that conforms to <code class="highlighter-rouge">AutoRangeMappable</code>, a protocol we use to mark the types that we want to extend. For each of those types we define an extension that implements the solution in Option 1 by iterating over all instance variable of type <code class="highlighter-rouge">SpreadSheetRange</code>.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{% for type in types.implementing.AutoRangeMappable %}
extension {{ type.name }} {
static let rangesCount = {{ type.instanceVariables.count }}
public var ranges: [SpreadSheetRange] {
return [
{% for variable in type.instanceVariables %}{% if variable.type.name == "SpreadSheetRange" %} {{ variable.name }}{% if not forloop.last %},{% endif %}{% endif %}
{% endfor %}
]
}
}
{% endfor %}
</code></pre></div></div>
<p>We only need to conform to <code class="highlighter-rouge">AutoRangeMappable</code>.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">struct</span> <span class="kt">RangeMapper</span><span class="p">:</span> <span class="kt">AutoRangeMappable</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">title</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="k">let</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="k">let</span> <span class="nv">attributes</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span>
<span class="kd">public</span> <span class="nf">init</span><span class="p">(</span><span class="nv">title</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">,</span> <span class="nv">description</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">,</span> <span class="nv">attributes</span><span class="p">:</span> <span class="kt">SpreadSheetRange</span><span class="p">)</span> <span class="p">{</span>
<span class="k">self</span><span class="o">.</span><span class="n">title</span> <span class="o">=</span> <span class="n">title</span>
<span class="k">self</span><span class="o">.</span><span class="n">description</span> <span class="o">=</span> <span class="n">description</span>
<span class="k">self</span><span class="o">.</span><span class="n">attributes</span> <span class="o">=</span> <span class="n">attributes</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Then Sourcery will generate the following extension based on the previous template.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">extension</span> <span class="kt">RangeMapper</span> <span class="p">{</span>
<span class="kd">static</span> <span class="k">let</span> <span class="nv">rangesCount</span> <span class="o">=</span> <span class="mi">3</span>
<span class="k">var</span> <span class="nv">ranges</span><span class="p">:</span> <span class="p">[</span><span class="kt">SpreadSheetRange</span><span class="p">]</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[</span>
<span class="n">title</span><span class="p">,</span>
<span class="n">description</span><span class="p">,</span>
<span class="n">attributes</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>If we want to add or remove an instance variable from <code class="highlighter-rouge">RangeMapper</code> all we have to do is run Sourcery and our extension will be automatically updated. You can add Sourcery as a build script and forget about it!</p>
<h3 id="to-sum-up">To sum up</h3>
<p>The idea of this blog post is to show a particular case where there are invariants that cannot be easily enforced by the language without having some sort of manual sync process and show possible solutions. I’d love to know other cases where you faced a similar situation and see how you resolved it.</p>
<blockquote>
<p>Check the <a href="/swift/2018/04/23/advanced-code-generation.html">following post</a> where I point out a bug in the Sourcery’s template and explain how to solve it and how to write more advanced templates.</p>
</blockquote>I wanted to write a bit about how we represent code invariants, specifically when those invariants cannot be easily expressed or enforced by the language, in this case the Swift programming language. When we talk about invariants Wikipedia’s definition explains it quite well:Message forwarding in Swift2017-06-03T06:00:00+00:002017-06-03T06:00:00+00:00http://guidomb.blog/swift/2017/06/03/message-forwarding-in-swift-8223dd31e009<p>One of the features I really liked about Ruby that is also present in Objective-C is the ability to forward messages.</p>
<p>In Ruby you can use the <code class="highlighter-rouge">Forwardable</code> module</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">LineItem</span>
<span class="kp">extend</span> <span class="no">Forwardable</span>
<span class="n">def_delegators</span> <span class="ss">:@product</span><span class="p">,</span> <span class="ss">:sku</span><span class="p">,</span> <span class="ss">:name</span><span class="p">,</span> <span class="ss">:description</span><span class="p">,</span> <span class="ss">:price</span>
<span class="nb">attr_reader</span> <span class="ss">:product</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">product</span><span class="p">)</span>
<span class="vi">@product</span> <span class="o">=</span> <span class="n">product</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>In Objective-C you can do something like</p>
<div class="language-objc highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">-</span> <span class="p">(</span><span class="n">id</span><span class="p">)</span><span class="n">negotiate</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">([</span><span class="n">someOtherObject</span> <span class="nf">respondsTo</span><span class="p">:</span><span class="k">@selector</span><span class="p">(</span><span class="n">negotiate</span><span class="p">)])</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[</span><span class="n">someOtherObject</span> <span class="nf">negotiate</span><span class="p">];</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">self</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<blockquote>
<p>If you want to know more about message forwarding in Objective-C you can read <a href="https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/ObjCRuntimeGuide/Articles/ocrtForwarding.html">Apple’s documentation</a></p>
</blockquote>
<p>This is one of those features that you rarely use but sometimes they help you avoid lots of boilerplate. In the case of Objective-C it leverages its run-time in order to dispatch messages dynamically. Something that cannot be done with pure Swift objects unless you made them inherit from <code class="highlighter-rouge">NSObject</code>.</p>
<p>Today I found myself in a situation when writing Swift code were message forwarding was a good solution to avoid some clutter and made me think about how could I accomplish something similar using pure Swift.</p>
<p>While working on <a href="https://github.com/guidomb/Portal">Portal</a>, a library that implements the <a href="https://guide.elm-lang.org/architecture/">Elm’s architecture</a> in Swift, I needed to implement the following protocol.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">protocol</span> <span class="kt">ApplicationRenderer</span> <span class="p">{</span>
<span class="kd">associatedtype</span> <span class="kt">MessageType</span>
<span class="kd">associatedtype</span> <span class="kt">RouteType</span><span class="p">:</span> <span class="kt">Route</span>
<span class="kd">associatedtype</span> <span class="kt">NavigatorType</span><span class="p">:</span> <span class="kt">Equatable</span>
<span class="kd">typealias</span> <span class="kt">ViewType</span> <span class="o">=</span> <span class="kt">View</span><span class="o"><</span><span class="kt">RouteType</span><span class="p">,</span> <span class="kt">MessageType</span><span class="p">,</span> <span class="kt">NavigatorType</span><span class="o">></span>
<span class="kd">typealias</span> <span class="kt">ActionType</span> <span class="o">=</span> <span class="kt">Action</span><span class="o"><</span><span class="kt">RouteType</span><span class="p">,</span> <span class="kt">MessageType</span><span class="o">></span>
<span class="k">var</span> <span class="nv">mailbox</span><span class="p">:</span> <span class="kt">Mailbox</span><span class="o"><</span><span class="kt">ActionType</span><span class="o">></span> <span class="p">{</span> <span class="k">get</span> <span class="p">}</span>
<span class="kd">func</span> <span class="nf">render</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="kt">ViewType</span><span class="p">,</span> <span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span>
<span class="kd">func</span> <span class="nf">present</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="kt">ViewType</span><span class="p">,</span> <span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span>
<span class="kd">func</span> <span class="nf">presentModal</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="kt">ViewType</span><span class="p">,</span> <span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span>
<span class="kd">func</span> <span class="nf">dismissCurrentNavigator</span><span class="p">(</span><span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span>
<span class="kd">func</span> <span class="nf">rewindCurrentNavigator</span><span class="p">(</span><span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The implementation that I was working on was the one responsible of rendering Portal’s view using UIKit. The conforming class is called <a href="https://github.com/guidomb/Portal/blob/d49495b517a78c50e4bd6eee62e9226f259d9208/Portal/UIKit/UIKitApplicationRenderer.swift">UIKitApplicationRenderer</a>. The tricky thing about implementing this protocol is that because of how UIKit works, all of its methods must be executed on the main thread. But the caller could already be running on the main thread or it may be running on a different thread.</p>
<p>My first attempt implementing this protocol looked something like this</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">func</span> <span class="nf">present</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="kt">ViewType</span><span class="p">,</span> <span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span> <span class="p">{</span>
<span class="kt">DispatchQueue</span><span class="o">.</span><span class="n">main</span><span class="o">.</span><span class="n">async</span> <span class="p">{</span>
<span class="c1">// code that presented the view</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The first thing that I wanted to change was to avoid dispatching work to the main dispatch queue if the method was already being called on the main thread. So I added the following helper method</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">fileprivate</span> <span class="kd">extension</span> <span class="kt">UIKitApplicationRenderer</span> <span class="p">{</span>
<span class="kd">fileprivate</span> <span class="kd">func</span> <span class="nf">executeInMainThread</span><span class="p">(</span><span class="n">_</span> <span class="nv">code</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="kt">Thread</span><span class="o">.</span><span class="n">isMainThread</span> <span class="p">{</span>
<span class="nf">code</span><span class="p">()</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="kt">DispatchQueue</span><span class="o">.</span><span class="n">main</span><span class="o">.</span><span class="n">async</span> <span class="p">{</span> <span class="nf">code</span><span class="p">()</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Then I made every method from the protocol call <code class="highlighter-rouge">executeInMainThread</code> . Every method looked something like</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">func</span> <span class="nf">present</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="kt">ViewType</span><span class="p">,</span> <span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span> <span class="p">{</span>
<span class="n">executeInMainThread</span> <span class="p">{</span>
<span class="c1">// code that presented the view</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>I could have call it a day and move to the next task. But I didn’t like how the code looked like. By having to use a closure to implement the actual logic on each method I was using self all over the place. I didn’t like that. That is when I thought that it would be nice to be able to forward methods.</p>
<p>What I ended up doing was creating a new class, <a href="https://github.com/guidomb/Portal/blob/d49495b517a78c50e4bd6eee62e9226f259d9208/Portal/UIKit/UIKitApplicationRenderer.swift#L112">MainThreadUIKitApplicationRenderer</a> that implemented the <code class="highlighter-rouge">ApplicationRenderer</code> protocol assuming that all of its method will be called on the main thread.</p>
<p>The next thing was updating <code class="highlighter-rouge">executeInMainThread</code> to make it simpler to pass a block of code to execute that would only contain one method call by making the closure parameter <code class="highlighter-rouge">@autoclosure</code>. This would have the benefit of avoiding to have to wrap each method invocation with a closure.</p>
<blockquote>
<p>If you want to know more about ``@autoclosure` and some cool ways of using it, check <a href="https://www.swiftbysundell.com/posts/using-autoclosure-when-designing-swift-apis">this</a> blog post.</p>
</blockquote>
<p>Finally all I needed to do was to add a forwardee property in UKitApplicationRenderer that would hold a reference to an object of type <code class="highlighter-rouge">MainThreadUIKitApplicationRenderer</code> to which all methods would be forwarded after making sure that the method invocation would be executed on the main thread. <code class="highlighter-rouge">UIKitApplicationRenderer</code> ended up looking something like</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">func</span> <span class="nf">render</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="kt">ViewType</span><span class="p">,</span> <span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span> <span class="p">{</span>
<span class="nf">executeInMainThread</span><span class="p">(</span><span class="k">self</span><span class="o">.</span><span class="n">forwardee</span><span class="o">.</span><span class="nf">render</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="n">view</span><span class="p">,</span> <span class="nv">completion</span><span class="p">:</span> <span class="n">completion</span><span class="p">))</span>
<span class="p">}</span>
<span class="kd">public</span> <span class="kd">func</span> <span class="nf">present</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="kt">ViewType</span><span class="p">,</span> <span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span> <span class="p">{</span>
<span class="nf">executeInMainThread</span><span class="p">(</span><span class="k">self</span><span class="o">.</span><span class="n">forwardee</span><span class="o">.</span><span class="nf">present</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="n">view</span><span class="p">,</span> <span class="nv">completion</span><span class="p">:</span> <span class="n">completion</span><span class="p">))</span>
<span class="p">}</span>
<span class="kd">public</span> <span class="kd">func</span> <span class="nf">presentModal</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="kt">ViewType</span><span class="p">,</span> <span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span> <span class="p">{</span>
<span class="nf">executeInMainThread</span><span class="p">(</span><span class="k">self</span><span class="o">.</span><span class="n">forwardee</span><span class="o">.</span><span class="nf">presentModal</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="n">view</span><span class="p">,</span> <span class="nv">completion</span><span class="p">:</span> <span class="n">completion</span><span class="p">))</span>
<span class="p">}</span>
<span class="kd">public</span> <span class="kd">func</span> <span class="nf">dismissCurrentNavigator</span><span class="p">(</span><span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span> <span class="p">{</span>
<span class="nf">executeInMainThread</span><span class="p">(</span><span class="k">self</span><span class="o">.</span><span class="n">forwardee</span><span class="o">.</span><span class="nf">dismissCurrentNavigator</span><span class="p">(</span><span class="nv">completion</span><span class="p">:</span> <span class="n">completion</span><span class="p">))</span>
<span class="p">}</span>
<span class="kd">public</span> <span class="kd">func</span> <span class="nf">rewindCurrentNavigator</span><span class="p">(</span><span class="nv">completion</span><span class="p">:</span> <span class="kd">@escaping</span> <span class="p">()</span> <span class="o">-></span> <span class="kt">Void</span><span class="p">)</span> <span class="p">{</span>
<span class="nf">executeInMainThread</span><span class="p">(</span><span class="k">self</span><span class="o">.</span><span class="n">forwardee</span><span class="o">.</span><span class="nf">rewindCurrentNavigator</span><span class="p">(</span><span class="nv">completion</span><span class="p">:</span> <span class="n">completion</span><span class="p">))</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Which I think reads better. Obviously compared to Objective-C or Ruby you still have to manually forward each method but that is something that may be solved by using some meta-programming with <a href="https://github.com/krzysztofzablocki/Sourcery">Sourcery</a>. I imagine that such solution would require defining a <code class="highlighter-rouge">Forwardable</code> protocol that could look something like</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">protocol</span> <span class="kt">Forwardable</span> <span class="p">{</span>
<span class="kd">associatedtype</span> <span class="kt">ForwardeeType</span>
<span class="k">var</span> <span class="nv">forwardee</span><span class="p">:</span> <span class="kt">ForwardeeType</span> <span class="p">{</span> <span class="k">get</span> <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Then Sourcery could check all classes that conform to Forwardable and by using <a href="https://cdn.rawgit.com/krzysztofzablocki/Sourcery/master/docs/writing-templates.html">Sourcery source annotations</a> you could tell it which is the protocol that you want to conform to and Sourcery could then generate an extension with all the boilerplate necessary to forward each method. I leave that as an exercise for the reader.</p>
<blockquote>
<p>There is a similar example in Sourcery documentation to implement decorators using metaprogramming. Check the example <a href="https://cdn.rawgit.com/krzysztofzablocki/Sourcery/master/docs/decorator.html">here</a>.</p>
</blockquote>
<p>Do you find this useful? Did you find yourself in a similar situation? What did you do then? Let me know by dropping a comment.</p>One of the features I really liked about Ruby that is also present in Objective-C is the ability to forward messages.Tips for better refactoring in Swift2017-05-31T06:00:00+00:002017-05-31T06:00:00+00:00http://guidomb.blog/swift/2017/05/31/tips-for-better-refactoring-in-swift-4a43c08961cd<p>One of the things I like the most about the Swift programming language is its type system. I think it has a good balance between <em>“main stream / industrial”</em> languages and more academic ones. Which makes it really approachable for new developers but also gives the more advanced ones a nice set of tools to be more productive.</p>
<p>What I also like is something that could be called <em>“typed-driven development”</em> (if such thing exists). Which basically boils down to define your types and function signatures and then trying to make every piece fit together until the compiler is happy. I found that by doing this, it makes implementing the actual logic quite straight forward.</p>
<p>Also having a type system and static checks come in really handy when you have to refactor code. The compiler becomes your first line of defense by pointing out all places that need you attention. Pieces that no longer fit together and need to be fixed.</p>
<p>Unfortunately not everything is as good as it could be. The Swift programming language is still very young and under heavy development. The compiler is not as mature as other compilers from languages that are ten or twenty years old. Nowadays it’s rare to see a compiler crash but not so long ago compiler crashes were a real issue that could throw away all productivity benefits Apple’s new language said it had.</p>
<p>Currently the mayor pain point regarding the compiler is diagnosis. Once you reach a fair amount of non-trivial code, very strange error messages will appear when you make a syntax error. Specially if you use lots of generics and type inference.</p>
<p>After a while of getting mad at the compiler and spending hours to understand what is the actual cause of the error (which usually requires trying to isolate the issue by reproducing the problem with the least amount of code possible), you learn how to read this cryptic and misleading error messages and know where to look at. You get a sort of intuition. Which is really hard to transfer to other team members.</p>
<h3 id="my-last-refactoring-adventure">My last refactoring adventure</h3>
<p>Yesterday I found myself in a similar situation while I was refactoring some code in <a href="http://github.com/guidomb/Portal">Portal</a>. I wanted to delete a class which implemented a certain protocol and I also wanted to change the method signatures in such protocol.</p>
<p>My first attempt was to delete the class, change the method signatures and make some class in the library implement the modified protocol. The idea was that the compiler would tell me all the places that stopped working and needed my attention. Which worked for the most part until I reached to a point of not being able to fix some errors pointed by the compiler.</p>
<p>The errors made no sense at all. Probably some bug in the compiler or a bad diagnosis. But after spending hours trying to fix it, I gave up. I discarded all my changes, shutdown my laptop and went home.</p>
<p>Today I decided to tackle the problem with a different approach. The first thing I did was to copy-paste the protocol I wanted to modify and made a second version of it called <code class="highlighter-rouge">ApplicationRenderer2</code>. Then I updated the method signatures. Everything compiled. I was just adding a new type.</p>
<p>Then I made a class in the library implement this new protocol. This required a little bit of work because some of the logic that was in the class that I intended to delete needed to be copy-pasted into this class. After half an hour the project compiled again.</p>
<p>Next step was replacing every dependency on <code class="highlighter-rouge">ApplicationRenderer</code> to <code class="highlighter-rouge">ApplicationRenderer2</code>. Which yielded lots of compiler error that where pretty simple to fix.</p>
<p>Finally I deleted the aforementioned class, hit build and checked that everything worked as expected. The last step was deleting <code class="highlighter-rouge">ApplicationRenderer</code> and renaming <code class="highlighter-rouge">ApplicationRenderer2</code> to ApplicationRenderer. The whole process took two hours.</p>
<h3 id="conclusion">Conclusion</h3>
<p>Thanks to only making <em>additive</em> changes and then only adding a <em>destructive</em> change at the end of the refactoring process made me accomplish my refactoring task much more effectively. This is a strategy I intend to use more in my future refactoring endeavors. At least until Swift gets better diagnosis. Give it a try and let me know what you think.</p>
<blockquote>
<p>If you are curious about the actual change that I made you can check commit <a href="https://github.com/guidomb/Portal/commit/757a2a6f4be997c50f5155e3b63bc894abfc3ec4">757a2a6f4be997c50f5155e3b63bc894abfc3ec4</a></p>
</blockquote>One of the things I like the most about the Swift programming language is its type system. I think it has a good balance between “main stream / industrial” languages and more academic ones. Which makes it really approachable for new developers but also gives the more advanced ones a nice set of tools to be more productive.Big brother, privacy issues and the quest of the Kurzgesagt2017-05-11T06:00:00+00:002017-05-11T06:00:00+00:00http://guidomb.blog/2017/05/11/big-brother-privacy-issues-and-the-quest-of-the-kurzgesagt-d90a6b37c635<p><img src="/assets/img/2017-05-11/bigbrother.jpeg" alt="big brother" /></p>
<p>I wanted to share a weird experience that happened to me last night. I was at the office at the end of the day when a couple of my coworkers decided to go hang out at the apartment of one of them. Who happened to live right next to the office. I still had some work that I had to finish. I told them that it would only take one hour.</p>
<p>After I finished my work I joined them. When I arrived they were watching <a href="https://vimeo.com/149851847">this music video</a> and they were talking about how cool the video was and the awesome illustrations and animations it has. This remind me of a YouTube channel that publishes cool videos also with awesome animations explaining different things, generally scientific stuff, in less than a couple of minutes. The problem was that I couldn’t remember the channel’s name. All I remembered was that it had a weird name and kind of looked like German but I wasn’t sure.</p>
<p>The funny thing was that a couple of weeks ago I was also thinking about this channel but again I couldn’t remember the name. I also remembered that it was quite a while since the last time I saw a video from that channel.</p>
<p>I didn’t have my computer with me so I opened the YouTube app on my phone and started to scroll down the Home screen thinking that a video from this channel would eventually appear. After a couple of minutes I couldn’t find any video from this channel. I then tried looking on the Subscriptions tabs, again with no luck. I thought it was weird. I was sure that I saw a video from this channel a couple of moths ago. Maybe there was a bug on the YouTube app or maybe I hit the unsubscribe button by mistake.</p>
<p>I was very frustrated. At this point I wanted to find the channel just not to feel defeated by the YouTube app. I grabbed my coworker’s laptop, opened Google and typed <strong>best animated videos scientific knowledge</strong>.</p>
<p><img src="/assets/img/2017-05-11/googlesearch.png" alt="Google search result" /></p>
<p class="image-caption"><em>Google search result</em></p>
<p>I clicked on <a href="https://www.geekwrapped.com/posts/youtube-science-rockstars-shows">The 50 Best YouTube Science Channels | GeekWrapped</a> and then I finally found it. The name of the channel was <a href="https://www.youtube.com/channel/UCsXVk37bltHxD1rDPwtNM8Q">Kurzgesagt — In a Nutshell</a>. I felt relieved knowing that I defeated evil YouTube on the quest of finding <strong>Kurzgesagt</strong>.</p>
<p><img src="/assets/img/2017-05-11/searchresult.png" alt="The 50 best YouTube science channels screenshot" /></p>
<p class="image-caption"><em>The 50 best YouTube science channels screenshot</em></p>
<p>First of all let me tell you that If you reached this far I’m quite surprised. I am either a decent writer or you really like to procrastinate. You might be wondering what’s all the fuss about finding a YouTube channel you couldn’t remember the name. This is where the “<em>cool</em>” part begins.</p>
<p>I went back home for dinner with my girlfriend. I was eating when I noticed that I’ve got a new email. I’m quite of a zero inbox freak so I couldn’t avoid doing a quick check on the Mail app. I opened the app and there it was, an email from YouTube with the subject <strong>Kurzgesagt — In a Nutshell: “The Last Light Before Eternal Darkness — White Dwarfs & Black Dwarfs” and more videos</strong>.</p>
<p><img src="/assets/img/2017-05-11/emailfromyoutube.png" alt="Email from YouTube" /></p>
<p class="image-caption"><em>Email from YouTube</em></p>
<p>I thought WTF YouTube. How could you send me an email about the latest Kurzgesagt videos after moths of not sending me any videos from them. Even after I spent a few minutes on the YouTube app searching for this channel without having any luck. Why now? I felt like big brother was watching me.</p>
<p>My girlfriend, who was already upset because I was checking my email at dinner saw that I was looking at my phone like something terrible had happened and asked me what was the big deal. I told her about what happened earlier that night and what a big coincidence was that I’ve got this email from YouTube.</p>
<p>I told her that I was thinking that Google / YouTube knew that I was looking for <strong>Kurzgesagt</strong> but I didn’t know how. I’ve never typed anything on my phone and didn’t visit the channel’s page in my phone after finding it on my coworker’s laptop. Remember I used my coworker’s laptop to search for the channel and I didn’t login into any service in that computer. So how could they know!!??</p>
<p>It could all be a coincidence but the conspiracy theorist in me refused to believe it. That is when my girlfriend said <em>“well, Google does know your current location and also knows your coworker’s location”</em>. That was actually true and the engineer in me was desperate to know how Google could know that I was looking for or might be interested in the <strong>Kurzgesagt</strong> channel. Here are the facts:</p>
<ol>
<li>
<p>Google knew my last location because I use Google on my computer and I gave it access to my location. The office is right next to my coworker’s apartment so the distance difference could be inside the accepted location error range.</p>
</li>
<li>
<p>I use quite a few Google apps on my phone and most of them have access to my location while I use them.</p>
</li>
<li>
<p>Google has access to my phone’s microphone. I explicitly have to press the mic button for Google to be able to record audio but technically they could turn my mic on without me noticing it. I choose to discard this option for now just because is too creepy and I don’t have strong evidence. Also it would require quite a sophisticated natural language processing to be able to infer that I was looking for the <strong>Kurzgesagt</strong> channel. I don’t think the tech is there yet.</p>
</li>
<li>
<p>They know when I opened the YouTube app and they know I scrolled quite a few pages both on the Home and Subscriptions screens. It is reasonable for them to infer that I was looking for something and because I didn’t watch any video nor I visit any of the channels’ page I failed to find what I was looking for.</p>
</li>
<li>
<p>Google has all this same data about my coworker’s behavior because we both use Google’s products.</p>
</li>
<li>
<p>Google could infer that I was hanging out with my coworkers because all of our GPS and interactions with Google products through our phones and laptops tell it we were in the same physical location.</p>
</li>
</ol>
<p>With all this information Google could infer that I might be interested in watching videos from <strong>Kurzgesagt</strong> because one of my coworkers visit the channel’s page on his computer while I was with him.</p>
<p>Again it could also be big coincidence. I cannot prove this but I am sure that Google has access to all this data, the computing power and the IA technology to be able to infer this and send me such email. It kind of freaks me out a little bit. I mean, I am software engineer. I know how this works and how such algorithms could be implemented. I know companies like Google probably do this things (and they probably do more creepy stuff) but when you start thinking about it and when you notice it in your daily life I couldn’t help feeling a little violated.</p>
<p>I’ll probably forget about it later this week but I think most people don’t realize about this sort of things. I think it’s something that should be discussed more often. The problem is that I don’t know what companies like Google know about me. We also know that governments and companies don’t hold the high moral grounds and have systematically violated people’s privacy rights in the name of profitability or <em>“for the sake of protecting their citizens”</em>.</p>
<p>The world we are living in it’s getting more complex every day. The line to separate what’s wrong or right, what is a product feature or a violation of people’s privacy it’s becoming more blurry. All I know is that we should be talking more about this and that as technical people how know more about this stuff and are the ones implementing this things we should start questioning the implications of our work.</p>Usage Driven Development2017-04-13T06:00:00+00:002017-04-13T06:00:00+00:00http://guidomb.blog/2017/04/13/usage-driven-development-a25cc5c50cd0<h2 id="a-brief-story-of-the-tools-i-wish-i-had">A brief story of the tools I wish I had</h2>
<p>Recently at <a href="http://wolox.co">Wolox</a> I’ve been working on a Swift library to help us develop applications more rapidly but also having a solid architecture that produces maintainable code. You’ll probably hear more about this in a future post.</p>
<p>Like most software projects, this one started as a rough idea that was rambling inside my head for a long time which eventually manifested as a prototype developed over a weekend. After a couple of days hacking, I had a first version of the library that was able to demonstrate its potential and kind of worked for basic iOS applications. So I decided to start refactoring the code, clean it up and start adding missing core features.</p>
<p>Soon I realized that I was starting to build a relatively complex code base that I knew would have to go over several iterations and refactor cycles until the APIs were polished enough to be able to be used by the widely range of application we build for our clients at Wolox. Although I was able to move fast and add feature after feature, I started to fear that I would hit a wall because I was (and still am) not writing any tests (yet).</p>
<p>Being a guy who always preached about <a href="https://en.wikipedia.org/wiki/Test-driven_development">TDD</a> and <a href="https://en.wikipedia.org/wiki/Behavior-driven_development">BDD</a>, this was a big issue for me. But I’ve also learned the hard way about committing too early to writing tests for rapidly-changing and unstable code. Tests are also code that has to be maintained and when you are designing a new API that works for you but that also needs to work for a variety of other cases that you cannot even imagine, the maintenance burden can slow you down really hard. That is why I decided that I was going to commit on testing after I was confident that the APIs were stable enough and no mayor changes were going to be needed in the foreseeable future.</p>
<p>Strong BDD advocates would tell me that I am crazy. That I should be driving the design of my API by writing test that would make me discover how those APIs should look like. Although I agree with such approach, I found that discovering the library’s API was much more effective by writing a real application. Just start working on a new app and write the interface you wish you had and then implement it. My <a href="http://blog.cleancoder.com/uncle-bob/2014/12/17/TheCyclesOfTDD.html">red-green-refactor</a> cycle became:</p>
<ol>
<li>
<p>Scaffold the new app feature, discover the API by watching all the compiler errors for types or function that don’t exist yet.</p>
</li>
<li>
<p>Implement those missing types or functions and make the project compile.</p>
</li>
<li>
<p>Once the feature is working see if you can refactor the code, generalize it or clean it. Make sure that the feature is still working.</p>
</li>
</ol>
<p>Applying this same approach in dynamic languages like Javascript or Ruby can be way harder and BDD in those language make more sense. You have to compensate not having a type-system and a compiler that tells you what you have broken after a refactor cycle. This is not the case for Swift. Sure, there are invariants that cannot be expressed in the type-system but in my experience so far those cases can be easily spotted by running the application and doing a quick check.</p>
<p>I know that once the code base reaches certain size and complexity this approach will no longer be valid but it helped me move faster at the beginning. That is why I am preparing for the next phase and the reason I am writing this blog post. Once I decide that the APIs are stable enough and more people starts depending on my library I need to start having strong code coverage, at least for the most important pieces and execution paths. This is were I would like to have better support from my tooling.</p>
<p>The following are a couple of tools I wish I had</p>
<p>A text editor / IDE plugin or chat bot that tells you which pieces should be tested, ordered from the highest to lowest priority. The priority is calculated based on the <strong>source file churn</strong> combined with <strong>runtime execution information</strong> of which code paths or functions are executed more often.</p>
<blockquote>
<p>Runtime information might be hard to collect without having significant performance impact. Maybe by instrumenting LLVM byte code by inserting functions call to a stats collector library every time there is a code jump or an execution branch, e.g., function calls, function returns, if statements. I am not sure if LLVM exposes a plugin mechanism that would make such extension relatively easy to implement</p>
</blockquote>
<p>Files with low churn that contain code that is executed more frequently means such code is stable and probably critical or that atleast is shared by most feature flows.</p>
<p>Files with high churn that contain code that is executed more frequently in (<em>in development builds</em>) means such code is actively being worked on and there for it might be wiser to test it once the API has been stabilized and the use case validated.</p>
<p>Crash reports is another good source of information to take into account and decide where and when we should put our attention.</p>
<p>It is extremely important that in order for such tool to be useful it must be integrated with your IDE and <strong>it should provide information in real time or when the user decides to “augment” her editing experience</strong>. Contextual information could be provided by the IDE’s UI when the user holds the course after a couple of seconds over a block of code. More frequent execution paths would be highlighted with a stronger color (by changing highlighted area alpha’s property) than less frequent ones. Execution paths that contain lines that are part of a crash report’s stack frame can be highlighted in red.</p>
<p>If the application is using an architecture like the <a href="https://guide.elm-lang.org/architecture/">Elm’s architecture</a> or something like <a href="http://redux.js.org/">Redux</a> then when a crash is detected the current state and the last <em>n</em> messages are attached to the crash report. The execution path that processes the messages and the state that generated the crash are highlighted by the IDE in real time when the crash occurs or when the user decides to debug a specific crash report.</p>
<p>Going one step further, each message that is processed by the application’s update function could be sent to a cloud service that processes the message in order to analyze and gather metrics from the code execution path. This provides real time runtime execution information without affecting performance and consuming user’s CPU because replicating the message is off-loaded to a background queue and the actual cost of instrumentation is paid by the backend server. This strategy increases data consumption and surely there are a lot of things that I haven’t though but should be technically possible now that Swift can run on Linux.</p>
<p>If someone wants to work in something like this or if such tools already exist please let me know. In the mean time I think I’ll have to settle with intuition and some basic code coverage reporting.</p>
<p>I think that a lot can be done by making developers more productive. People think that in a near feature AI would take over developers jobs and machines will start writing code by themselves. I think such future is still pretty far away. What I do think is that the progress in computing power and AI could be used to develop better and smarter tooling which provides contextual real time information to help developers make better decision. What do you think?</p>A brief story of the tools I wish I hadA short guide for IoT startups2017-01-31T06:00:00+00:002017-01-31T06:00:00+00:00http://guidomb.blog/iot/2017/01/31/a-short-guide-for-iot-startups-a96e13485fc9<p>I was asked to give a talk targeted to investor that didn’t have experience investing in IoT startups. Although I’m far from being an expert on the subject I think I have some notes I can share after 3 years of working on a hardware startup.</p>
<p>Most of the following applies to a <em>subset</em> of hardware startups, nowadays called <strong>IoT</strong> or <em>internet of things</em>. Which in most cases referrers to consumer electronic devices that are either connected to the internet or to some sort of network of devices. Things like smart light bulbs, smart lighters or connected devices to grow plants and vegetables. Although there are processes and advice that applies to IoT startups and also more “<em>old-school</em>” hardware startups. Is not the same to build a smart light bulb as having to build an autonomous robot that can navigate an unknown location.</p>
<h3 id="software-startups-vs-iot-startups">Software startups v.s. IoT startups</h3>
<p>Although most of the processes and methodologies involved in developing a product on a software startups also apply to IoT startups, there are some differences that need to be noticed.</p>
<p>I think that is important for someone that is entering the world of hardware startups for the first time to know the different stages involved in developing a hardware product and the different risks that they may face during this process.</p>
<p><em>I recently found another blog post that also talks about the process of building a hardware product. After reading this one take a look at <a href="https://backchannel.com/how-to-build-a-hard-tech-startup-4028d22f2c91#.khrghbt85">this</a> other one.</em></p>
<h3 id="hardware-startup-product-stages">Hardware startup product stages</h3>
<p>Most hardware startups and in particular IoT startups go through a set of stages when they develop a product. Each stage has different goals and it only makes sense to move forward to the next stage once all or most of the hypothesis for that stage have been validated. This stages usually are:</p>
<ul>
<li>
<p>Idea</p>
</li>
<li>
<p>Proof of concept</p>
</li>
<li>
<p>Prototyping</p>
</li>
<li>
<p>D.F.M. or design for manufacture</p>
</li>
<li>
<p>Production</p>
</li>
</ul>
<h3 id="idea">Idea</h3>
<p>This stage consists mostly of working on your idea, defining the value proposition and your main hypothesis. By the end of this stage you should have a clear idea of what problem you are trying to solve and a rough idea of how you will solve that problem. It is recommended to work on some design concepts, at least at a very high level of how the product would look like and it is useful to have a 3D render of the product. This helps a lot to communicate and visualize your product in order to talk with potential angel investor or to recruit some co-founder or first employees. It is also a good idea to create a landing page and start getting some feedback from potential users.</p>
<h3 id="proof-of-concept">Proof of concept</h3>
<p>This stage allow you to validate that a rough concept can be made and that the technology is mature enough, at least to demonstrate your product’s core functionality. At this stage engineers usually use development boards and prototyping tools like <a href="https://www.arduino.cc/">Arduino</a>, <a href="https://www.raspberrypi.org/">Raspberry PI</a> or integrated circuit boards that already come with things like a micro-controller, <a href="https://en.wikipedia.org/wiki/Inertial_measurement_unit">IMU sensors</a>, Bluetooth or WiFi chips. Aesthetics and cost is generally not a concern at this stage.</p>
<p>By the end of this stage you can look for seed investment to be able to go through the next stage. Hopefully you have a working proof of concept that can demonstrate your product’s core value proposition and with a strong pitch you can convince investor to bet on your startup.</p>
<p><em>Sometimes it happens that the technology is available but the product cannot be launched because it either fails to satisfy other constrains like size, battery life or production cost is too high. Being able to reach this stage does not mean that all risks have been mitigated. Usually is the other way around, the most risk adverse part is yet to come.</em></p>
<h3 id="prototyping">Prototyping</h3>
<p>After validating that the product’s core functionality can be built you move to the prototyping stage. This stage may consiste of several iterations of the product where you take into consideration production costs, the product’s form factor, aesthetics, materials, etc.</p>
<p>Generally speaking this stage involves the design and manufacture of custom <a href="https://en.wikipedia.org/wiki/Printed_circuit_board">PCB</a> and a case where that PCB will be held. Engineers usually produce 5 to 20 pieces of each design variation or prototype to test its performance, robustness, different materials and the general user experience. It its recommended at this stage to test the different prototypes with “real” users to try to get as much feedback as possible.</p>
<p>This is a critical a stage for an IoT startup because at this stage is when you lock down how the final product will look like, how it will behave and its core features. That is why is so important not to move forward until you have validated that the prototype you have built has a real value proposition that users are willing to pay for. Also at this stage you will have a pretty good estimate of the final production cost. Although that will be refined in the next stage, at this moment you can estimate how much money you will need for the following stages and you can commit to a retail price.</p>
<h3 id="dfm-or-design-for-manufacture">D.F.M. or design for manufacture</h3>
<p>Here is where you start preparing the design to be able to be manufactured at medium or large scale. This process usually involves talking with factories to know if they are capable of building your product in a cost effective way. This stage may require minor tweaks in the design or changing some components or materials in order to make the manufacturing processes easier. Here is where you also learn if your product will require certain certifications in order to be sold in your target market.</p>
<p>It is recommended to hire an external consultant with experience on manufacturing if this is the first time your team is going through this process. Making sure this stage is well executed is critical for the success of the product. A lot of hardware startups underestimate this stage thinking that going to production from the prototyping stage is just a matter of making more units. It is not. The manufacturing process required to build 20 units is completely different from building 100, 1,000 or 10,000.</p>
<p>Finally, here is when you can lock down the final production costs and know for sure what your retail price and revenue will be. This is the safest stage to start taking pre-orders either by accepting them through your web page or by launching a crowdfunding campaign in sites like <a href="https://www.kickstarter.com/">Kickstarter</a> or <a href="https://www.indiegogo.com">Indiegogo</a>. Take into account that in order to move to the next stage you will need to be at a point of having to call the factories and tell them to start building your product which will require more cash upfront. Some startups raise another investment round here, something between a seed and series A round. Another options is to launch a crowdfunding campaign then, if the campaign is successful, you may still want to raise a round but in this case (depending of how successful the campaign was) you will stand on a different position.</p>
<h3 id="production">Production</h3>
<p>This stage varies in complexity depending on the amount of units that will be produced. Different orders of magnitude usually require different production techniques. In general at this stage you have to coordinate with the factory or factories. Once this stage has been reached you cannot make any changes to the product except maybe some minor tweaks here and there to make the production process smoother or to avoid any possible performance problems. Things like replacing non-critical electronic components, minor layout changes on the P.C.B. or shaving a couples of millimeters from the case’s mold.</p>
<p>Once you are sure everything is ready you will need to commit on tooling. This is why this stage requires quite a lot of cash upfront. In general, most IoT product have some sort of <a href="https://en.wikipedia.org/wiki/Injection_moulding">injection molding</a> case. Which means that a factory will need to produce a mold first. This process varies depending on the complexity of the case, the technique used and the materials but it usually takes 3 to 4 weeks. Once the mold is ready a pre-production run is made, around 150 units. This gives you the chance to fix minor issues, shave some rough edges and do some intensive end-to-end testing.</p>
<p>Some startups use this period and the pre-production units with beta tester who usually are your evangelizer, first users that are really engaged with your product. Getting feedback at this stage is also really important because you can polish some processes like unboxing for example. Check that all the installation instructions are clear. If you provide some sort of software like a mobile application, you can check that the first user flow goes as smooth as possible, that users are able to connect with your device and know how to use it.</p>
<h3 id="how-much-money-do-you-need-to-reach-mvp">How much money do you need to reach M.V.P?</h3>
<p>Another key difference between pure software startups and IoT startups is the amount of cash they need and the time it takes to reach M.V.P. Each of the previous stage requires a significant amount of money and iteration times are way longer than on a pure software product.</p>
<h3 id="cash-and-time-per-stage">Cash and time per stage</h3>
<ul>
<li>
<p><strong>Idea:</strong> This could take <strong>1 to 2 months</strong> and maybe around <strong>5k USD</strong> if you need to hire a free lancer to do some sketches, 3D renders or a landing page.</p>
</li>
<li>
<p><strong>Proof of concept:</strong> Around <strong>15k USD</strong> and <strong>1 to 3 months</strong>.</p>
</li>
<li>
<p><strong>Prototyping:</strong> Assuming 3 prototypes iterations with custom P.C.B. and some sort of casing using <a href="https://en.wikipedia.org/wiki/Numerical_control">CNC</a> or 3D printing. Around <strong>150k USD</strong> and <strong>6 to 8 months</strong>.</p>
</li>
<li>
<p><strong>D.F.M.:</strong> Around <strong>3 months</strong> and <strong>60k to 90k USD</strong></p>
</li>
<li>
<p><strong>Production:</strong> Depending on the amount of units, but lets say a first production run of 500 to 1000 units. Around <strong>60k USD</strong> and <strong>2 months</strong>.</p>
</li>
</ul>
<p>For what I observed so far and IoT startups usually takes <strong>at least 18 months</strong> to go from idea to production, assuming all stages are executed correctly and there any mayor problems. Again this varies a lot depending on the complexity of the product but <strong>2 years</strong> and <strong>350k to 500k USD</strong> investment seems to be the bare minimum.</p>
<h3 id="what-about-risks">What about risks?</h3>
<p>An IoT startups is subject to the same risks most startups face, being the most important building the wrong product and running out of money. But if you come from a pure software startup background there are some things that you should probably consider.</p>
<h3 id="certifications-patents-and-taxes">Certifications, patents and taxes</h3>
<p>Most probably if your product requires some sort of wireless connectivity like Bluetooth or WiFi most likely you will have to certify your antenna design. If your product doesn’t have the required certifications you won’t be able to sell it in some markets. This regulations depend on each market, selling a product in Europe may require different certifications than in the U.S. Another option instead of having to go through the certification process is to use pre-certified modules. For example there are modules that you can include in your custom circuit that already include a Bluetooth chip and an antenna. This modules have already been certified by the Bluetooth consortium. This may increase your unit cost but frees you of having to certify your product. Which certifications are needed depends on the specific of your product but take into account that they take time and money. Is usually safe to add <strong>3 weeks</strong> and <strong>5k to 15k USD</strong>.</p>
<p>Another thing to consider is patents, which also applies to pure software startups. This could add another <strong>12k to 20k USD</strong> to your budget. Same as with certifications, they change depending on where you want to sell your product. There are agreements between countries, for example in the case of the EU you only apply for a patent that is valid on all members of the EU.</p>
<p>Also when building a hardware product, chances are that you would require different components or parts that you may not find in the country you are doing the R&D. Depending on the regulations of the country you are based on, importing this components may incur in import feeds that could significantly increase your development costs. Not to mention the bureaucracy and delays.</p>
<h3 id="cash-flow">Cash flow</h3>
<p>Running out of cash is something that all startups have to be concerned of but this problem accelerates with hardware startups due to the amount of cash needed for product development. You not only have to pay for engineers salaries and external consultants fees but you have to pay for tools, materials and everything needed to build your prototypes. Not to mention the required tooling once you are in the production stage.</p>
<p>The risk of running out of cash increases when you are about to make your first production run. Factories usually don’t give you credit, you have to pay them up front both for the tooling costs and the production cost per unit. You pay for product that you will be able to sell and cash in at least 3 months in advanced. Other things can add up quickly, like import / export fees, storage and shipping fees.</p>
<p>After you build a relation with your factory they may give you credit and some times they do it the first time you work with them if they like your product or see some strategic value of working with you.</p>
<h3 id="design-and-manufacturing-issues">Design and manufacturing issues</h3>
<p>A couple of things regarding product design that shouldn’t be overlooked:</p>
<ul>
<li>
<p>Products that have mechanic and moving parts are usually harder to build and add more risk to the manufacturing process.</p>
</li>
<li>
<p>Things like shock-proof and water-proof is not something to take lightly. If your product needs be to water-proof and this is the first time that you are working on a product like this, hire an external consultant at least to review your design and help you with manufacturing. There are quite a few cases of startups failing at this promises adding significant delays or even having to recall product that didn’t work in the environment there were supposed to.</p>
</li>
</ul>
<h3 id="fulfillment-of-physical-goods">Fulfillment of physical goods</h3>
<p>Again, another area that most people that come from a pure software startup background don’t know is dealing with an actual physical product.</p>
<p>Building a product is just one of the many things you need to do. After the product is built, you need to sell it. This is where you need to think which are your distribution channels. Again, this changes depending on the product but if you are selling a consumer electronic type of product you probably want to be on the shelf of most well known retailers. Getting access to big retail chains is not easy. They usually get products that sell well and are proven. They tend to place big orders and expect them to be fulfilled on time. They even put huge penalties for each day you fail to deliver. You’d better be prepared and have all your processes fine-tuned before going for the retailers like Walmart, Target or Best Buy, if your target market is the U.S.</p>
<p>That is why startups usually start by first taking pre-orders on their site or through a crowdfunding campaign, then they move to online orders only and they start working on distribution deals. Take into account that if your product needs to be sold on local shops, going shop by shop across a whole country for usually low volume orders could be close to impossible for startups with low budget and a small team. That is why selling your product to distributors alleviates this problem because they take bigger orders and they deal with managing stock and getting the product to their network of retailers. Remember that managing product stock costs money, you have to store your product somewhere and the logistics behind distribution it not easy.</p>
<p>The downside of retailers and distributors is that they take a significant share of the retail price, usually between <strong>40% and 60% of the retail price.</strong> Selling your product through your website gives you higher margins and chances are that at the beginning you won’t be taken a lot of orders. You can deal with shipping yourself and once it gets out-of-hand you can hire a service that helps you with order fulfillment. Some factories even have deals with fulfillment centers (in the U.S.) and they can ship the product directly to them. No need of having to maintain stock.</p>
<h3 id="intellectual-property">Intellectual property</h3>
<p>Intellectual property is a concern specially when dealing with Chinese factories. There are tons of cases of factories coping and selling the product they are making for their customer. After all they know how to make it. Sad news is that there isn’t much you can do about it and to be honest is not worth it.</p>
<p>Chinese factories won’t sign you and <a href="https://en.wikipedia.org/wiki/Non-disclosure_agreement">NDA</a> or a non-compete, unless your are Apple. Instead of wasting a lot of money on lawyers and patents the smart thing to do is building a company around a product that does not depend on the revenue that it gets by selling the hardware only. Also hardware is getting cheaper and cheaper and production costs are decreasing year after year. Chances are that your product will get commoditized eventually.</p>
<p>That is why it is recommended to use the hardware as a vehicle to unlock certain features and develop other revenue streams. For example if your hardware cannot do much without a companion mobile application, then factories will need to build their own copy cat of your app. Sometimes the technology behind the product is not that cutting edge or innovative but the way it is applied and how is marketed it is. Building a strong brand and providing extra features or value to the hardware through a software component is the safest way to protect you from being copied.</p>
<h3 id="revenue-streams">Revenue streams</h3>
<p>One of the most critical parts of a hardware startup to ensure its survival is to know your revenue streams and your costs. Getting the right retail price is really important and is a decision that should be taken with actual information. Pricing is not trivial and users may be willing to pay more o less depending on factors such as how they perceive your brand, the type of product you are selling or the value you are providing.</p>
<p>A good rule of thumb to set a base retail price for your product is to <strong>multiply the unit cost by 4</strong>. This includes actual production cost which you can infer from the <a href="https://en.wikipedia.org/wiki/Bill_of_materials">BoM or bill of materials</a> plus shipping and packaging costs. This rule gives you enough margin to pay for distributors, retailers, payment processing feeds and some refunds.</p>
<p>As mentioned before when we talked about intellectual property, it is important that you build other sources of revenue. Even old-school traditional hardware companies are adding services, usually through software and in many cases startups don’t make money by selling hardware. They use hardware as way to enable their business and the real money comes from software. It is even a common practice for startups to sell their hardware at a price just to meet the costs or even under the production costs because that is way to improve adoption and increase the entrance barrier for competition. This of course is possible if the hardware enables them to activate other sources of revenue.</p>
<h3 id="conclusion">Conclusion</h3>
<p>I know, this looks a little bit scary but let me tell you that going through this process is really fun. As an entrepreneur you will need to learn how to wear a lot of new hats. As a investor entering the hardware world for the first time, is good to know the difference between building a hardware and a software product and set the right expectations.</p>
<p>The hardware community is awesome and there is a lot of material out there. Most founders that have gone through this process would love to share their experience and help newcomers. Don’t be afraid of asking.</p>I was asked to give a talk targeted to investor that didn’t have experience investing in IoT startups. Although I’m far from being an expert on the subject I think I have some notes I can share after 3 years of working on a hardware startup.Avoiding bugs and improving documentation through the type system2017-01-06T06:00:00+00:002017-01-06T06:00:00+00:00http://guidomb.blog/swift/2017/01/06/avoiding-bugs-and-improving-documentation-through-the-type-system-61d2695aceb6<p><em>Always try to reduce functions and methods domain. Improve documentation by using narrower types which enforce pre-conditions. Give the compiler more information. Watch <a href="https://www.youtube.com/watch?v=DU7Je1rdXk0">this</a> video.</em></p>
<p>Today I was reviewing a pull request and I found the following piece of code that was added</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">final</span> <span class="kd">class</span> <span class="kt">UserSessionService</span> <span class="p">{</span>
<span class="kd">public</span> <span class="nf">init</span><span class="p">(</span><span class="n">withTimeout</span> <span class="nv">timeout</span><span class="p">:</span> <span class="kt">NSTimeInterval</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Initialization logic</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Although the code was working fine and followed all of our best practices I felt that there was something there that could still be improved. The first thing that triggered this thought was actually a “cosmetic” issue. I thought, what if the parameter’s label, instead of being <strong>withTimeout</strong> is changed to with. That change would imply that creating a new instance of <strong>UserSerssionService</strong> would change from</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">userSessionService</span> <span class="o">=</span> <span class="kt">UserSessionService</span><span class="p">(</span><span class="nv">withTimeout</span><span class="p">:</span> <span class="mi">10</span><span class="p">)</span>
</code></pre></div></div>
<p>to</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">userSessionService</span> <span class="o">=</span> <span class="kt">UserSessionService</span><span class="p">(</span><span class="nv">with</span><span class="p">:</span> <span class="mi">10</span><span class="p">)</span>
</code></pre></div></div>
<p>Clearly not a good idea. Although it is less verbose, it makes understanding what is actually happening way harder. What does it mean to create a user session service <strong>“with 10”</strong>. The original solution at least clarifies that the given number is a timeout. It reads as, “create a user session service with timeout 10”. But the problem with it is that is not much clearer than that. A timeout for what? My next thought was to suggest changing the initializer to</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="nf">init</span><span class="p">(</span><span class="n">withRequestTimeout</span> <span class="nv">requestTimeout</span><span class="p">:</span> <span class="kt">NSTimeInterval</span><span class="p">)</span>
</code></pre></div></div>
<p>Which clarifies that timeout is related to the requests this service makes. <strong>UserSessionService</strong> exposes methods to login and sign up. The thing with this solution is that it’s worse than the original one regarding the length of the label. Obviously not a big deal, but this is what was bothering me in the first place.</p>
<p>If you read <a href="https://swift.org/documentation/api-design-guidelines/#naming">Swift’s API design guidelines</a> didn’t help much either. Except from some tips like</p>
<blockquote>
<p>func move(from start: Point, to end: Point)
Choose parameter names to serve documentation. Even though parameter names do not appear at a function or method’s point of use, they play an important explanatory role.</p>
</blockquote>
<p>and</p>
<blockquote>
<p>func move(from start: Point, to end: Point)</p>
</blockquote>
<p>I realized that the problem was actually with type of the parameter, <strong>requestTimeout</strong>. If the type had a more descriptive name the method signature could read better and be less verbose. For example</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">typealias</span> <span class="kt">RequestTimeout</span> <span class="o">=</span> <span class="kt">NSTimeInterval</span>
<span class="c1">/// Return a `UserSessionRepository` that limits login and sign up</span>
<span class="c1">/// requests to the given `requestTimeout`.</span>
<span class="kd">public</span> <span class="nf">init</span><span class="p">(</span><span class="n">with</span> <span class="nv">requestTimeout</span><span class="p">:</span> <span class="kt">RequestTimeout</span><span class="p">)</span>
</code></pre></div></div>
<p>This “felt” better but it had the same issue as not being clear enough when used. Which led me to change the type alias to a concrete type.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">struct</span> <span class="kt">RequestTimeout</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">value</span><span class="p">:</span> <span class="kt">NSTimeInterval</span>
<span class="nf">init</span><span class="p">(</span><span class="n">_</span> <span class="nv">value</span><span class="p">:</span> <span class="kt">NSTimeInterval</span><span class="p">)</span> <span class="p">{</span>
<span class="k">self</span><span class="o">.</span><span class="n">value</span> <span class="o">=</span> <span class="n">value</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">public</span> <span class="kd">final</span> <span class="kd">class</span> <span class="kt">UserSessionService</span> <span class="p">{</span>
<span class="kd">public</span> <span class="nf">init</span><span class="p">(</span><span class="n">with</span> <span class="nv">timeout</span><span class="p">:</span> <span class="kt">RequestTimeout</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// initialization logic</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Which implies the following change when creating a new user session service</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">userSessionService</span> <span class="o">=</span> <span class="kt">UserSessionService</span><span class="p">(</span><span class="nv">with</span><span class="p">:</span> <span class="kt">RequestTimeout</span><span class="p">(</span><span class="mi">10</span><span class="p">))</span>
</code></pre></div></div>
<p>Although the amount of “verbosity” remains almost the same, this solution felt superior. I’ve realized that by introducing a type I was improving readability and documentation and also making sure that whoever created a new instance of <strong>UserSessionService</strong> got a compiler error if the appropriate type wasn’t used. Having a more “narrower” type helps you catch error earlier. For example when a parameter is not a literal value in the code but something that is passed to you. Sure, in this case you could create a <strong>RequestTimeout</strong> with any given <strong>NSTimeInterval</strong> (which is just a type alias to Double). But having to do that explicitly makes you think twice about what you are doing and makes it more clear in code reviews in order for reviewers to double check if the type promotion is correct.</p>
<p>It didn’t end there. Then I thought, “wait, I can create a <strong>RequestTimeout</strong> given any number, clearly -45.345 is not a valid <strong>RequestTimeout</strong>”. The easiest way to solve this is by using <a href="https://developer.apple.com/library/content/documentation/Swift/Conceptual/Swift_Programming_Language/Initialization.html">fail-able initializers</a>.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">struct</span> <span class="kt">RequestTimeout</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">value</span><span class="p">:</span> <span class="kt">Double</span>
<span class="nf">init</span><span class="p">?(</span><span class="n">_</span> <span class="nv">value</span><span class="p">:</span> <span class="kt">Double</span><span class="p">)</span> <span class="p">{</span>
<span class="k">guard</span> <span class="n">value</span> <span class="o">></span> <span class="mi">0</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="kc">nil</span> <span class="p">}</span>
<span class="k">self</span><span class="o">.</span><span class="n">value</span> <span class="o">=</span> <span class="n">value</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Now we need to safely unwrap the option.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="k">let</span> <span class="nv">requestTimeout</span> <span class="o">=</span> <span class="kt">RequestTimeout</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">userSessionService</span> <span class="o">=</span> <span class="kt">UserSessionService</span><span class="p">(</span><span class="nv">with</span><span class="p">:</span> <span class="n">requestTimeout</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Depending on how you are creating the <strong>RequestTimeout</strong> object and where that initialization takes place, using <em>if-let</em> syntax may not be necessary. For example, lets say to you only need to create the user session service once and you do it in the application’s delegate.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">class</span> <span class="kt">AppDelegate</span><span class="p">:</span> <span class="kt">UIResponder</span><span class="p">,</span> <span class="kt">UIApplicationDelegate</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">userSessionService</span> <span class="o">=</span> <span class="kt">UserSessionService</span><span class="p">(</span><span class="nv">with</span><span class="p">:</span> <span class="kt">RequestTimeout</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span><span class="o">!</span><span class="p">)</span>
<span class="c1">// The rest of the UIApplicationDelegate methods</span>
<span class="p">}</span>
</code></pre></div></div>
<p>In such case I think that explicitly unwrapping the <strong>RequestTimeout</strong> value is OK because worst case scenario you’ll get a crash the moment you open the application. If you have at least one test, that error will get caught really early. Not as good as a compiler error but good enough.</p>
<p>I was about to move on but I decided to show all this to my co-workers to get their opinion. All of them agreed with this but one pointed out that it could be improved a little bit by communicating the time unit used for the request timeout. Because <strong>RequestTimeout(10)</strong> does not saying anything about whether the request timeout is in minutes, seconds, milliseconds or whatever. Because the request timeout is in seconds I changed the its initializer to</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">struct</span> <span class="kt">RequestTimeout</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">value</span><span class="p">:</span> <span class="kt">Double</span>
<span class="nf">init</span><span class="p">?(</span><span class="n">seconds</span> <span class="nv">value</span><span class="p">:</span> <span class="kt">Double</span><span class="p">)</span> <span class="p">{</span>
<span class="k">guard</span> <span class="n">value</span> <span class="o">></span> <span class="mi">0</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="kc">nil</span> <span class="p">}</span>
<span class="k">self</span><span class="o">.</span><span class="n">value</span> <span class="o">=</span> <span class="n">value</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>which reads like</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="k">let</span> <span class="nv">requestTimeout</span> <span class="o">=</span> <span class="kt">RequestTimeout</span><span class="p">(</span><span class="nv">seconds</span><span class="p">:</span> <span class="mi">10</span><span class="p">)</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">userSessionService</span> <span class="o">=</span> <span class="kt">UserSessionService</span><span class="p">(</span><span class="nv">with</span><span class="p">:</span> <span class="n">requestTimeout</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="conclusion">Conclusion</h3>
<p>I think that providing documentation through the type system is the best way of designing code. Being able to empower the compiler by giving it more information helps you catch future bugs. I came to this conclusion by trying to improve a minor style issue which as a matter of fact it was pretty subjective (an probably only fueled by my <a href="https://en.wikipedia.org/wiki/Obsessive%E2%80%93compulsive_disorder">OCD</a> with code). That led to further challenging my assumptions and the end result was better code.</p>
<p>After all this review process and once I had the solution I felt comfortable with, I decided to look about this on the internet and found the following video explaining this same concept from <a href="twitter.com/chriseidhof">Chris Eidhof</a> and <a href="https://twitter.com/floriankugler">Florian Kugler</a>.</p>
<center><iframe width="560" height="315" src="https://www.youtube.com/embed/DU7Je1rdXk0" frameborder="0" allowfullscreen=""></iframe></center>
<p>Also found this Twitter thread by <a href="https://twitter.com/olebegemann">Ole Begemann</a></p>
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">The same goes for other quantities. Your animation API has a duration parameter? Don’t use TimeInterval, use Measurement<UnitDuration>.</p>— Ole Begemann (@olebegemann) <a href="https://twitter.com/olebegemann/status/816312282560995328?ref_src=twsrc%5Etfw">January 3, 2017</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>and a his <a href="https://oleb.net/blog/2016/07/measurements-and-units/">blog post</a> about the Foundation’s Measurement type.</p>Always try to reduce functions and methods domain. Improve documentation by using narrower types which enforce pre-conditions. Give the compiler more information. Watch this video.Syrmo’s backend stack: Why we choose Parse as our BaaS solution?2016-11-18T06:00:00+00:002016-11-18T06:00:00+00:00http://guidomb.blog/syrmo/2016/11/18/syrmos-backend-stack-why-we-choose-parse-as-our-baas-solution-251cc2716296<p><em>This is the first post of the series about the technology behind Syrmo. The choices we made, the good, the bad and the ugly.</em></p>
<h3 id="tl-dr">tl; dr</h3>
<p><em>Parse is a really good solution for prototyping and getting an MVP quickly into the market. It has an active community behind it and it is well documented. It serves well as a CRUD application with authentication and authorization out of the box. It has nice SDKs for iOS, Android and Javascript. It is really easy to deploy and the mayor IaaS and PaaS providers have support for it.</em></p>
<p><em>If you have a small team, tight deadlines and an application that does not require backend business logic. Parse is a really good option.If you do have some backend business logic you can use AWS Lambda.</em></p>
<p><em>Parse + AWS Lambda = 💙 💚 💛</em></p>
<h3 id="a-little-bit-of-background">A little bit of background</h3>
<p>When we started working on <a href="http://syrmo.com">Syrmo</a>, a performance tracker for skateboarders, we didn’t start working on the end user iOS app first. There were a lot of unknowns and technical problems that we needed to solve before we can design how the end user experience will be like using Syrmo’s mobile app. Things like:</p>
<ul>
<li>
<p>How the Bluetooth LE protocol works and if it was well suited for our particular use case.</p>
</li>
<li>
<p>Validate that the sensors we were planning to use were accurate enough to gather the date we needed in order to be a able infer different metrics about a skateboard trick.</p>
</li>
<li>
<p>Learn about 3D rending technology and choose a cross-platform stack that will let us generate 3D animated replay of each skateboard trick based on the data recorded using our tracking device.</p>
</li>
<li>
<p>Develop a set of high-performant algorithms that will run on the tracking device and could detect when a user performed a skateboard trick discarding “noise” data.</p>
</li>
</ul>
<p>Of course, as any other bootstrapping startup out there, we needed to figure out all this with an under staffed team, in the least amount of time and spending as few money as possible. Working on the backend was (and still is) really down in the list of priorities.</p>
<p>Once we tackled down most of our critical unknowns and started to show progress we began working on the end user mobile app. After a couple of discussions on what Syrmo’s main value proposition was, we realized that its core feature needed to work without an internet connection. Skateboarders should be able to record a new skating session without needing to be connected to the internet. Obviously they need to be connected if they want to share their latest cool trick with their friends, check what their favorites skaters are doing or look for the hottest skateboarding spots near their location.</p>
<p>Did I say that we were understaffed? Well, for a long period of time I was the only software engineer in the team. I didn’t want to spend time with the hassles of developing and maintaining a backend application. All I needed was CRUD application and an easy way to access the models over HTTP. There was almost no business logic that needed to run in the backed.</p>
<p>After doing some research I found <a href="http://parse.com">Parse</a>. It had everything I needed. It was a backend as a service. No need to provision machines, setup databases, design REST API or develop a networking layer. Parse had a</p>
<ul>
<li>
<p>Mobile SDK for Android and iOS.</p>
</li>
<li>
<p>Blob storage.</p>
</li>
<li>
<p>A database model similar to MongoDB (which I had experience with).</p>
</li>
<li>
<p>Authentication and authorization out of the box.</p>
</li>
<li>
<p>Ability to execute custom business logic using <a href="https://parseplatform.github.io/docs/cloudcode/guide/">CloudCode</a>.</p>
</li>
</ul>
<p>I did a quick prototype to test that all the features advertised by Parse worked as expected. After a couple of days I was really happy with the developer experience and what I could a accomplish in such a short time. Nevertheless I was still hesitant about Parse. I had a lot of experience with Ruby on Rails. I knew I could solve any problem with it.</p>
<p>Using a new technology is always fun and challenging but it comes with the associated cost of learning it. No matter how promising the new technology is, you will move slower first. Also when you look at the docs and the tutorials they always show you the happy paths. But what happens when you need to do something that nobody did before and when you cannot find a similar example? Ruby on Rails has 10 plus years of development and a huge community. Chances are that somebody already found the solution to your problem.</p>
<p>By the time we needed to start integrating a backend to our app the team got bigger. Two developers started to work with me on the iOS app. The catch was that they didn’t have any experience on iOS nor Ruby on Rails. Which made my decision between Parse or Ruby on Rails quite easy. I decided to go with Parse.</p>
<p>Basically I didn’t have the time to teach them both iOS and Rails. Also, as any other startup, we want to minimize our costs and engineers are one of the biggest source of cost. Not only because you have to pay them a salary, but also because they need to learn about the product, how the business works and in this case learn a new technology. Not to mention that when you add more people to a team you gain new problems. People’s personal issues, coordination, communication, egos, etc. All this can slow you down.</p>
<h3 id="about-our-experience-with-parse">About our experience with Parse</h3>
<p>After a couple of months working with Parse, we were pretty happy with it. Most of the issues we encountered were related with how the data model works. If you come with traditional SQL database background there are some things that may look weird to you. For example how you model relations between object and the performance cost you may face. Sometimes is better to denormalize data to avoid joins. In Parse, joins are performed at the application level, meaning that they are not supported by the database, which means that in order to fetch relations you need several round trips to the database. This translates into higher latency and memory consumption.</p>
<p>Another problem that we have to solve was related with the offline feature. As I mentioned before Syrmo’s core feature needs to work without an internet connection. We need to be able to store each trick’s movement data and all the session’s metadata (like location and duration) in the user’s local database and then sync it back with the server when a connection is available. Parse offers a way to save objects into the client’s local datastore (they call it <em>object pinning</em>). But there is a limitation with objects that have relations. If those relations were not persisted in the backend, saving the parent object won’t save the child objects. There is another feature that lets you “save objects eventually”. This feature saves object asynchronously and if the operation fails it stores the object inside an on-disk cache. The problem with this feature is that the cache has a fixed size (10MB according to the docs) that when reached it deletes old unsaved objects. Also once the objects are saved they are removed from the cache.</p>
<p>We ended up rolling out our solution to store objects on disk first and then using the save eventually feature to sync the local data with the backend. This solution is still far from ideal and it was the main source of bugs and complexity. A lot of bugs started to appear once we decided to add the possibility to edit and delete this objects. I could write a separate blog post about this (and probably will), but I can tell you now that maintaining consistency between the application’s local store and the backend while being fault tolerant and making sure that you won’t loose any data, is not a task to take lightly. There are companies which have dedicated products to solve this problems. You need to design your application with this constrains in mind from the start. Otherwise, adding offline support after the fact is going to be hard.</p>
<p>Bottom line, if you only need to store some non-critical objects the features provided by Parse are good enough but I wouldn’t recommend it for any advanced offline-first application.</p>
<p><em>If your application requires real time collaboration and offline support you may find Firebase or Realm’s mobile platform a better option. I don’t have production experience with them. I only made some toy applications with Firebase but their collaboration and offline feature seems to be better designed than Parse’s.</em></p>
<h3 id="parse-shutdown">Parse shutdown</h3>
<p>Everything was going smoothly until <a href="http://blog.parse.com/announcements/moving-on/">Parse announced they were shutting down the service</a>. This was a big surprise and something that I couldn’t anticipate. One of the reason I picked Parse in favor of other BaaS was that it had Facebook behind it.</p>
<p>The timing couldn’t be worse. We had already invested significant amount of time on Parse. Making a transition to another service or writing our custom backend would incur in significant delays. I immediately started to evaluate different alternatives. After some research my options were:</p>
<ol>
<li>
<p>Keep using Parse and eventually transition to the open source version.</p>
</li>
<li>
<p>Move to Firebase.</p>
</li>
<li>
<p>Move to AWS Mobile platform (a combination of Labmda, Dynamo, Cognito, S3).</p>
</li>
<li>
<p>Move to a custom Ruby on Rails application.</p>
</li>
</ol>
<p>I decided not to go with AWS Mobile, due to the complexity of managing AWS services and that some glue code was required in order to make everything work. Also with AWS there is no simple way of having a local development environment. (Parse didn’t have a local development enviroment but it was super easy and free to a get one instance per developer).</p>
<p>Then I spent two weeks reading Firebase’s documentation and making a <a href="https://github.com/guidomb/FirebaseFeed">super simple Twitter-like app</a>. I really liked what I could do with Firebase. Even more their realtime features are awesome. I knew that Firebase had everything I needed to implement all required features for Syrmo. But again, migrating to Firebase would require time that we didn’t have.</p>
<p>Before jumping to a new boat I decided to give Parse a new chance. A couple of months after the shutdown announcement, the community appeared to be really responsive about the open source version of Parse. A lot of issues were being solved, new features got added and the documentation and migration plans were really good. I was still reluctant to having to maintain my own server, at least at this stage but I said to myself: “If the migration to the open source version of Parse works as smooth as the docs say and after a month there aren’t any mayor issues, we’ll keep using Parse”. And so it was.</p>
<p>We now have our open source version of Parse server deployed to AWS Beanstalk. Let me tell you that the actual <a href="https://aws.amazon.com/blogs/mobile/how-to-set-up-parse-server-on-aws-using-aws-elastic-beanstalk/">setup</a> took me less than half an hour.</p>
<h3 id="what-about-parse-cloudcode">What about Parse CloudCode?</h3>
<p>So far everything was working fine but there was an issue that needed to be solved. One of the feature that I liked about Parse was the possibility to execute custom code. This is a pretty important feature if you need to run some business logic in the backend. Although most of our logic is driven by the client, there are some features that are way simpler and more efficient to implement in the backend. Such as the user’s feed. Similar to Twitter’s feed, users can have a list of skate tricks shared by their friends in real time. Details on how we implemented this will be shared in a future blog post.</p>
<p>We needed a simple way to execute logic in the server. Something simple that will make a couple of queries to Parse, create some files in S3 and then create some models and save them back to Parse. I wanted something simple and didn’t want to deal with provisioning, designing REST endpoints, and choosing web frameworks. I just wanted to run a simple function, like I could do with CloudCode.</p>
<p>Thankfully there is a service that does exactly that (and even more), <a href="https://docs.aws.amazon.com/lambda/latest/dg/welcome.html">AWS Lambda</a>. Using <a href="https://github.com/ParsePlatform/Parse-SDK-JS">Parse’s Javascript SDK</a> the integration between AWS Lambda and our server was a piece of cake.</p>
<h3 id="conclusion">Conclusion</h3>
<p>I could say that I am pretty happy with the choice I’ve made. Parse helped us focus on our client application and by using AWS Lambda we have the flexibility of writing custom code without having to worry about all the hassles of setting up a web service.</p>
<p>Would I choose Parse again? Maybe. Depending on what needs to be done. If a need to make a product as fast as I can where most of the logic is driven by the client, then yes. I already know how Parse works and I can be productive from day one. But there are other BaaS and open source backends out there that may be as good. If you want real time features and don’t want to deal with provisioning, <a href="https://firebase.google.com">Firebase</a> might be a good choice. Similar to Firebase but open source is <a href="https://horizon.io/">Horizon</a>, which like Parse decided to shutdown their paid service and release it as an open source project. Finally, <a href="https://realm.io/products/realm-mobile-platform/">Realm</a> has recently announced a service that helps you build real time apps.</p>
<p>Not all products and teams are the same. There is no rule of thumb but I am sure that if I am working on a new product with a small team and time-to-market is a concern, not having to worry about a backend it’s a pretty good idea.</p>
<p>Something that I learned over the last two years of working on Syrmo is that at an early stage startup you don’t have to worry to much about scalability, vendor lock-in, programming language, architecture design and all those things we software engineers love to deal with. All it matters is getting a product that works and proves that it has a valid value proposition and that there is a market willing to pay for that product.</p>
<p>Focus on discovering what the right product is and what your business model is. Choose the technology that enables you to accomplish that in the fastest way possible. If that means using technology that you are productive with but is not “cool enough”, go for it. Making shortcuts at this stage is OK. If you do a good job, then you will have the opportunity to sit down a make more informed decisions on which is the best technology and architecture for your product.</p>
<p>I our case Parse gave us the chance to avoid having to think too much about our backend and focus our resources to develop a good app experience.</p>
<p><em>If you have experience using Parse in production, feel free to share your experience in the comments or reach out to me on Twitter @guidomb</em></p>This is the first post of the series about the technology behind Syrmo. The choices we made, the good, the bad and the ugly.