Mike Slinn's Blog 2020-09-08T13:02:59-04:00 https://mslinn.github.io/blog Mike Slinn mslinn@gmail.com Bash Script to Create a New Jekyll Post 2020-08-16T00:00:00-04:00 https://mslinn.github.io/blog/2020/08/16/new-jekyll-post <p> I use <a href="https://jekyllrb.com/" target="_blank" rel="nofollow">Jekyll</a> to build this website. Some material is published as articles, some as blog posts. I wrote a script called <code>newPost</code> that creates a new draft blog post with SEO considerations. SEO rankings are improved when the description and title tag are neither too long nor too short. </p> <h2 id="usage">Sample Usage</h2> <p> Here is an example of how I used it on 2020-08-16: </p> <pre data-lt-active='false'> <span class="unselectable">$ </span>_bin/newPost Post Title (30-60 characters): ______________________________123456789012345678901234567890 This is a test of newPost for the greater good 46 characters, excellent! Post Description (30-60 characters): ____________________________________________________________123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890 I wish newPost used some sort of web service to generate an SEO-optimized description 85 characters, excellent! Post Categories (comma delimited): Post Tags (comma delimited): Post Keywords (comma delimited): Goofiness, Silliness Banner image (bg_ .jpg): </pre> <p> For the above example the generated file is called <code> _drafts/2020-08-16-this-is-a-test-of-newpost-for-the-greater-good.html</code>. When you are happy with the new posting, move it from <code>_drafts</code> to <code>_posts</code> like this: </p> <pre data-lt-active='false'> <span class="unselectable">$ </span>mv _drafts/2020-08-16-new-jekyll-post.html _posts/ </pre> <h3 id="generated">Generated Posting</h3> <p> Here is the generated file: </p> <pre data-lt-active='false'> --- categories: [] description: I wish newPost used some sort of web service to generate an SEO-optimized description image: keywords: [Goofiness, Silliness] last_modified_at: 2020-08-16 layout: blog title: This is a test of newPost for the greater good tags: [] --- </pre> <h2 id="error">Error Handling</h2> <p> The script checks the length of the title and the posting description for SEO purposes. If either of these are too long or too short, the script allows the user to edit their input over and over until they get it right. For example, here you can see that at first the user just types in <code>xx</code> for the title, then they provide a string that is too long, then they edit it until it has an acceptable length: </p> <pre data-lt-active='false'> <span class="unselectable">$ </span>_bin/newPost Post Title (30-60 characters): ______________________________123456789012345678901234567890 xx 28 characters too short, please edit Post Title (30-60 characters): ______________________________123456789012345678901234567890 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 14 characters too long, please edit Post Title (30-60 characters): ______________________________:123456789012345678901234567890 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 59 characters, excellent! </pre> <h2 id="code">Source Code</h2> <p>This is the source code for the <code>newPost</code> bash script.</p> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/eb59ce9149ff70e04bbd20b6ccd3cd2c.js"> </script> Converting All Images in a Website to webp Format 2020-08-15T00:00:00-04:00 https://mslinn.github.io/blog/2020/08/15/converting-all-images-to-webp-format <p> I first launched this website in 1996. Since then, it has been re-incarnated using many different technologies. Presently I use <a href="https://jekyllrb.com/" target="_blank" rel="nofollow">Jekyll</a> to assemble the site, then push the image to a web-enabled AWS S3 bucket that is edge-cached by an AWS CloudFront distribution. </p> <p> Until yesterday, the site contained images with a mixture of image formats. I decided to convert them all to the new <a href="https://developers.google.com/speed/webp" target="_blank" rel='nofollow'><code>webp</code></a> format. Because there are hundreds of images in over 120 web pages, I wrote a bash script called <code>toWebP</code> to do the work. This posting provides the <code>toWebP</code> script plus instructions on how you could use it for your website. </p> <p> The script converts image types <code>gif</code>, <code>jpg</code>, <code>jpeg</code>, <code>png</code>, <code>tif</code>, and <code>tiff</code>. It also modifies the HTML pages, CSS and SCSS that reference those images. </p> <p> The conversions are set for maximum fidelity (lossless where possible), and maximum compression. This means the images look great and load quickly. </p> <h3>Caveat</h3> <p> The script assumes that all images are local to your website, which makes sense because the converted images need to be stored, and local storage is the only sensible option. It renames all references to images in HTML, CSS and SCSS files to <code>webp</code> format. If the images are remote (for example, on a CDN), they are not converted, but the image file types in the HTML, CSS and SCSS are adjusted anyway. I suppose I could fix the script, but I don't need to do that for myself. If someone needs that feature, go ahead and enhance the script... and please provide me the enhanced script, so I can update this blog posting. </p> <h2 id="prerequisites">Prerequisites</h2> <p> You need to install the WebP package.<br> </p> <h3 id="mac">Mac</h3> <p> Use <a href="https://formulae.brew.sh/formula/webp" target="_blank" rel='nofollow'>Homebrew</a> or <a href="https://ports.macports.org/?search=webp&search_by=name" target="_blank" rel='nofollow'>Macports</a>. </p> <h3 id="ubuntu">Ubuntu (this is the default Linux distribution for Windows Subsystem for Linux)</h3> <p>At a shell prompt type:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>yes | sudo apt install webp</pre> <h2 id="running">Running <span class="code">toWebp</span></h2> <p> The program may emit warnings when it runs. Those warnings can be safely ignored. </p> <p> Hopefully, your website is managed by git. I suggest that you commit your work before running the script. That way if something goes wrong you just have to type <code>git stash</code> to return your website to its previous state. </p> <h3 id="usage">Usage</h3> <p>The general form of the command to convert all images and modify the HTML pages that they are referenced from is:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>toWebp &lt;directoryName></pre> <h3 id="examples">Examples</h3> <p>To convert the website (images, html, scss & css) rooted at the current directory, type:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>toWebp .</pre> <p>To convert the website called <code>mySite</code> rooted under your home directory, type:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>toWebp ~/mySite</pre> <p>To just convert 1 specific image to <code>webp</code>, type:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>toWebp images/blah.jpg</pre> <h2 id="gist">Gist Containing the <code>toWebP</code> Bash Script.</h2> <p>Put this file in one of the directories on your <code>PATH</code>, for example <code>/usr/local/bin</code>: <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/d87f13c921456a21070c4d96366c6778.js"> </script> <h3 id="chmod">Make it Executable</h3> <p> Remember to make the <code>toWebp</code> script executable before trying to use it: </p> <pre data-lt-active='false'><span class="unselectable">$ </span>chmod a+x /usr/local/bin/toWebp</pre> Dotty (Scala 3 Preview) Presentation at Hopper, Montreal 2019-11-28T00:00:00-05:00 https://mslinn.github.io/blog/2019/11/28/dotty-scala-3-preview <div style=""> <picture> <source srcset="/blog/images/dottyLambda_690x388.webp" type="image/webp"> <source srcset="/blog/images/dottyLambda_690x388.png" type="image/png"> <img src="/blog/images/dottyLambda_690x388.png" title="Mike Slinn presents" class=" liImg " alt="Mike Slinn presents" /> </picture> </div> <div style="text-align: center"> <p> Yesterday I presented <a href="https://www.meetup.com/lambda-montreal/events/266306046/" target="_blank" rel="nofollow">Dotty (Scala 3 Preview)</a> to Lambda Montreal. </p> <p> The slides are <a href="https://www.slideshare.net/mslinn/dotty-scala-3-preview" target="_blank" rel="nofollow">here</a>. </p> <p> The code is <a href="https://github.com/mslinn/dotty-example-project/" target="_blank" rel="nofollow">here</a>. </p> <p> The video recording is <a href="https://www.youtube.com/watch?v=7S68TY0S2e0" target="_blank" rel="nofollow">here</a>. </p> </div> <div style="text-align: center;"> <picture> <source srcset="/blog/images/lambdaMontreal.webp" type="image/webp"> <source srcset="/blog/images/lambdaMontreal.png" type="image/png"> <img src="/blog/images/lambdaMontreal.png" title="Mike Slinn presents" class="center quartersize liImg2 rounded shadow" alt="Mike Slinn presents" /> </picture> </div> A Hybrid Machine Learning / Personality Simulation Platform 2019-10-24T00:00:00-04:00 https://mslinn.github.io/blog/2019/10/24/hybrid-ml-simulation <div style="text-align: center;"> <a href="https://www.meetup.com/MTL-Machine-Learning/events/265039754/" target="_blank" rel="nofollow"><picture> <source srcset="/blog/images/mtlMLconference.webp" type="image/webp"> <source srcset="/blog/images/mtlMLconference.png" type="image/png"> <img src="/blog/images/mtlMLconference.png" class="center liImg2 rounded shadow" /> </picture></a> </div> <p> Yesterday I presented <a href="https://www.meetup.com/MTL-Machine-Learning/events/265039754/" target="_blank" rel="nofollow"> &ldquo;EmpathyWorks: A Hybrid Machine Learning / Personality Simulation Platform&rdquo; </a> to the <a href="https://www.meetup.com/MTL-Machine-Learning/events/265039754/" target="_blank" rel="nofollow">Fall 2019 Montreal Machine Learning Mini-Conference.</a> </p> <p> The slides are <a href="https://www.slideshare.net/mslinn/empathyworks-towards-an-eventbased-simulationml-hybrid-platform" target="_blank" rel="nofollow">here</a>. The video recording is <a href="https://youtu.be/PiDsiyJIMmo" target="_blank" rel="nofollow">here</a>. </p> <div style="text-align: center"> <div style="display: inline-block; margin: 0.5em; vertical-align: top;"> <picture> <source srcset="/assets/images/robotCircle207x207.webp" type="image/webp"> <source srcset="/assets/images/robotCircle207x207.png" type="image/png"> <img src="/assets/images/robotCircle207x207.png" title="Mike Slinn presents" class=" liImg " alt="Mike Slinn presents" /> </picture> </div> <div style="display: inline-block; margin: 0.5em; vertical-align: top;"> <picture> <source srcset="/blog/images/montrealMachineLearningMeetup.webp" type="image/webp"> <source srcset="/blog/images/montrealMachineLearningMeetup.png" type="image/png"> <img src="/blog/images/montrealMachineLearningMeetup.png" title="Montreal Machine Learning Meetup" class=" quartersize liImg2 rounded shadow" style="margin-left: 3em; margin-top: 3em;" alt="Montreal Machine Learning Meetup" /> </picture> </div> </div> Decentralized Ponytails 2018-09-13T00:00:00-04:00 https://mslinn.github.io/blog/2018/09/13/decentralized-ponytails <p> I&rsquo;d like to point out the similarity of the early days of the open-source movement with today&rsquo;s decentralized blockchain movement. </p> <p> Open-source software was brought to mainstream attention during the last technology bubble at the end of the last millennium. The open-source software movement had a loyal cadre of zealots who believed that their cause would overcome any need for a business case. Sun Microsystems was the hardware company whose servers powered the Internet, and their software included the Java programming language, plus many other important networking-related products. Sun's slogan was &ldquo;The network is the computer&rdquo;. </p> <div style="text-align: center;"> <picture> <source srcset="/assets/images/Sun-Logo_225x99.webp" type="image/webp"> <source srcset="/assets/images/Sun-Logo_225x99.png" type="image/png"> <img src="/assets/images/Sun-Logo_225x99.png" title="Sun Microsystems logo" class="center quartersize liImg2 rounded shadow" style="padding: 1em" alt="Sun Microsystems logo" /> </picture> </div> <p> Jonathan Schwartz, the CEO of Sun Microsystems was one of the open-source zealots. He was famous for his ponytail. Unfortunately, zealotry and dogma is bad for business, and as a result Sun Microsystems is no longer with us. </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/jonathanSchwartz.webp" type="image/webp"> <source srcset="/blog/images/jonathanSchwartz.png" type="image/png"> <img src="/blog/images/jonathanSchwartz.png" title="Jonathan Schwartz and his ponytail" class="center liImg rounded shadow" alt="Jonathan Schwartz and his ponytail" /> </picture> </div> <p> Eventually companies like <a href="https://redhat.com" target="_blank" rel='nofollow'>Red Hat</a> developed solid business models for open-source software, but that took years to develop. Today we see many companies attempting using decentralized blockchain technology to create cryptocurrencies, other token-based economies, and evangelizing decentralized dogma without a solid business case. Most of these ventures will die a horrible death, and the investors will get nothing. It will take years for solid business models based on decentralization to be proven. </p> <p> Mr. Schwartz's ponytail was the fashion statement that fueled the YouTube parody below. For background, <a href="https://en.wikipedia.org/wiki/Scott_McNealy" target="_blank" rel='nofollow'>Scott McNealy</a> was the previous CEO at Sun Microsystems. </p> <iframe width="690" height="388" src="https://www.youtube.com/embed/5r3JSciJf5M" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen class="rounded shadow liImg"></iframe> <p> Full disclosure: I also had a ponytail in 2008, and for a few years I had my Sun Spark 2 workstation at home. </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/mikeclose3.webp" type="image/webp"> <source srcset="/blog/images/mikeclose3.png" type="image/png"> <img src="/blog/images/mikeclose3.png" title="Mike Slinn and his ponytail back in 2008" class="center quartersize liImg2 rounded shadow" alt="Mike Slinn and his ponytail back in 2008" /> </picture> </div> <p> Here is a transcription of the video, which I paraphrased for clarity:</p> </p> <blockquote> <p><b>Steve Gilmore:</b> Hi, this is Steve Gilmore and this is a video special edition of the Gilmore gang. I'm here with Jonathan Schwartz. It's a great pleasure - it's been a long long time coming - I haven't seen Jonathan for quite a while. Jonathan Schwartz, who is the president and CEO of Sun Microsystems, agreed to sit down for the first time in three or four years and talk about what's going on with Sun. I wanna start, Jonathan, by thanking you for joining us. </p> <p><b>Jonathan Schwartz:</b> Thank you for having me, Steve. It has been a long time, nice to see you. </p> <p><b>Steve Gilmore:</b> So, you know there's been a lot of turmoil on Wall Street as I know you know. </p> <p><b>Jonathan Schwartz:</b> Yes. </p> <p><b>Steve Gilmore:</b> What's your take on that? </p> <p><b>Jonathan Schwartz:</b> Well, I think that it's a cyclical thing as you know Sun was been very prepared for this because we took <a href="https://www.thestreet.com/story/10334514/1/pipe-deal-shines-up-sun.html" target="_blank" rel='nofollow'>three quarters of a billion dollars off of KKR</a> a few years ago so that's in our war chest and I think that we're in a very good position moving forward, Steve. </p> <p><b>Steve Gilmore:</b> Ahh, and specifically what are you doing? </p> <p><b>Jonathan Schwartz:</b> Well, what we're doing is as you know Sun has always been very proactive in the open-source movement. I know you're very familiar with that. What we want to do to help our customers during this very difficult time is keep up with that trend of open-source. So I'm actually very pleased to announce, Steve, that Sun is starting a new open-source initiative. We're going to be releasing the source code to my ponytail as open-source, Steve. </p> <p><b>Steve Gilmore:</b> How's that going to help the situation? </p> <p><b>Jonathan Schwartz:</b> It's open-source, Steve. </p> <p><b>Steve Gilmore:</b> Yeah. </p> <p><b>Jonathan Schwartz:</b> It's my ponytail, Steve. </p> <p><b>Steve Gilmore:</b> You know, we've had this conversation in the past. </p> <p><b>Jonathan Schwartz:</b> Yes. </p> <p><b>Steve Gilmore:</b> Something is open-source, fine, and you get a lot of adoption, you get a lot of exposure in the in the marketing arena... </p> <p><b>Jonathan Schwartz:</b> Yes. </p> <p><b>Steve Gilmore:</b> ... but how do you make money on your ponytail? </p> <p><b>Jonathan Schwartz:</b> Well, what we're going to be doing is releasing my ponytail as open-source. So what we're hoping is that our developers take my ponytail and develop some kind of revenue stream with my ponytail. Did I mention this is open-source, Steve? </p> <p><b>Steve Gilmore:</b> Yeah, so how do you open-source a ponytail? What does that mean? </p> <p><b>Jonathan Schwartz:</b> Well, basically what we do is, we will have some of our best and brightest engineers here at Sun go through my ponytail and find out the unique attributes about what makes my ponytail so successful in the valley. And what we've done Steve, we've open-sourced my ponytail, Steve. </p> <p><b>Steve Gilmore:</b> Jonathan, you're just repeating this over and over again; it doesn't necessarily arrive at a business model. </p> <p><b>Jonathan Schwartz:</b> Umm, Steve? </p> <p><b>Steve Gilmore:</b> Yeah. </p> <p><b>Jonathan Schwartz:</b> We're gonna take my ponytail right and make it open-source. Now I know that this is a big concept for you but I really think it's a game changer, Steve. </p> <p><b>Steve Gilmore:</b> So, who do you see as your competition in the open-source ponytail arena? </p> <p><b>Jonathan Schwartz:</b> I think that we pretty much have it locked up. I don't see anybody who can compete with Sun Microsystems, when it comes to the open-source ponytail market. I think that we're in very good shape, Steve. How much do you miss Scott McNealy? </p> <p><b>Steve Gilmore:</b> Right now, a lot. </p> <p><b>Jonathan Schwartz:</b> Not nearly as much as I do, Steve. </p> <p><b>Steve Gilmore:</b> Okay, so what are you gonna do about, uh, you've laid off a lot of people in the last few months. </p> <p><b>Jonathan Schwartz:</b> Yes, it's going well in fact we have another round of layoffs coming. Once the ponytail is released into the wild, we'll be releasing the team that open-sourced my ponytail, Steve. </p> <p><b>Steve Gilmore:</b> Well you know I have to say that I was hoping for something a little bit more visionary from you Jonathan. </p> <p><b>Jonathan Schwartz:</b> Well I think that this is quite visionary. I don't think that IBM will be releasing a ponytail, and if they did it certainly wouldn't be open-source, Steve. We are very, very excited about our open-source ponytail program if you want you can go to <code>sunmicrosystems.com/ponytail</code>. Any other questions, Steve? </p> <p><b>Steve Gilmore:</b> Yeah I got one that I hope will be a stumper for you, which is a do you see the relationship between your open-source ponytail strategy and micro-messaging as popularized by Twitter? </p> <p><b>Jonathan Schwartz:</b> I'd like to open the pipe to the ponytail. Except as you know, Twitter is not exactly handling XMPP correctly at this time. I don't see the correlation between an open-source ponytail, and a closed-off micro-blogging system. Steve, I think that the ponytail is much bigger than Twitter. </p> <p><b>Steve Gilmore:</b> And the business model again? </p> <p><b>Jonathan Schwartz:</b> Let me see this real slowly and clearly: OPEN. SOURCE. PONYTAIL. </p> <p><b>Steve Gilmore:</b> This has been Jonathan Schwartz along with me, Steve Gilmore. Good luck, Jon. </p> <p><b>Jonathan Schwartz:</b> Thank you Steve. God, I miss McNealy. Think it's going to work? </p> <p><b>Steve Gilmore:</b> No. </p> </blockquote> IBM Personality Insights 2018-08-29T00:00:00-04:00 https://mslinn.github.io/blog/2018/08/29/personality-assessment <p> <a href="https://www.empathyworks.ai" rel="nofollow">empathyworks.ai</a> describes the original research I've done on modeling personality and behavior of individuals and groups since 2007. For me, the work has been both therapeutic and insightful, and I now have a better basis for understanding myself and others as a result. </p> <p> Recently, an IBM employee pointed me to <a href="https://console.bluemix.net/docs/services/personality-insights" rel="nofollow">IBM Personality Insights</a>, and I eagerly visited the site. The <a href="https://console.bluemix.net/docs/services/personality-insights/science.html#science" rel="nofollow">scientific basis</a> for the results is interesting. I am greatly interested in the analysis that IBM Personality Insights provided of the blog posting I wrote on my birthday last year. I also submitted a <a href="/blog/2008/04/28/cult-of-software-god.html">short humorous posting</a> I wrote ten years ago for analysis. </p> <h2 id="salt">A Modern Horoscope?</h2> <div style=""> <picture> <source srcset="/blog/images/zodiac_690x690.webp" type="image/webp"> <source srcset="/blog/images/zodiac_690x690.png" type="image/png"> <img src="/blog/images/zodiac_690x690.png" title="Horoscope" class=" liImg2 rounded shadow" alt="Horoscope" /> </picture> </div> <p> I think the results from IBM Personality Insights are about as accurate as a horoscope. <a href="https://www.quora.com/How-accurate-is-IBMs-Watson-Personality-Insights-application/answer/Abhishek-Srivastava-198" rel="nofollow">Abhishek Srivastava&rsquo;s Quora posting</a> of November 18, 2016, expresses this well. </p> <blockquote> I think most answers here have missed the point of IBM Watson’s personality insight service. It is a NLP based approach to find scores of individuals on some well-established scales in psychology like Big 5, Basic Humans Values and Needs. So, when those numbers are arrived using those standard questionnaires or through text analysis done by Watson, they are pretty close statistically. In that respect Watson is definitely very accurate. As far as interpretation of those results are concerned, that’s beyond the purview of current scope of Watson and rather is a question for psychologist. If I was a team member at IBM Watson, I would have actually not given the interpretation and would rather just give the scores and let people interpret them. </blockquote> <p> Furthermore, until a peer review of IBM Personality Insights concludes that the results are well-founded, I would not be comfortable using the technology for decision-making. </p> <p> To be fair, <a href="https://www.news.vcu.edu/article/An_untested_foundation_A_VCU_study_finds_that_many_published" rel="nofollow">a recent examination</a> of nearly 350 published psychological experiments found that 42% failed to show that they were based on a valid foundation of empirical evidence, suggesting that a wide swath of psychological science is based on an untested foundation. With that in mind, let's see what this modern horoscope serves up! </p> <h2 id="highLevel">High Level Results</h2> <p> Results were fairly consistent between the two blog postings. When I concatenated the two posts, the longer post dominated. <span class='diff'>Differences between the two personality assessments are shown this way.</span> <span class="comment">My comments are shown this way.</span> </p> <table class="table table_striped table_cell_top table_cell_vspace table_cell_justify" width="100%"> <tr> <th width="50%">Birthday Posting Summary</th> <th width="50%"><a href="/blog/2008/04/28/cult-of-software-god.html">Humorous Posting</a> Summary</th> </tr> <tr> <td> The results in JSON format are <a href="/blog/ibmPersonality.json">here</a>. </td> <td> The results in JSON format are <a href="/blog/ibmPersonality2.json">here</a>. </td> </tr> <tr> <td> You are shrewd and <span class='diff'>skeptical</span>. </td> <td> You are shrewd<span class='diff'>, inner-directed and guarded</span>. </td> </tr> <tr> <td> You are philosophical: you are open to and intrigued by new ideas and love to explore them. You are independent: you have a strong desire to have time to yourself. <span class='diff'>And you are authority-challenging: you prefer to challenge authority and traditional values to help bring about positive changes.</span> <td> You are philosophical: you are open to and intrigued by new ideas and love to explore them. You are independent: you have a strong desire to have time to yourself. <span class='diff'>And you are solemn: you are generally serious and do not joke much.</span> <span class="comment">I guess IBM did not like my humor!</span> </td> </tr> <tr> <td> Your choices are driven by a desire for <span class='diff'>discovery</span> <span class='comment'>often true</span>. </td> <td> Your choices are driven by a desire for <span class='diff'>organization</span> <span class='comment'>often true</span>. </td> </tr> <tr> <td> You are relatively unconcerned with both <span class='diff'>tradition</span> and taking pleasure in life. <span class='diff'>You care more about making your own path than following what others have done. And you prefer activities with a purpose greater than just personal enjoyment.</span> </td> <td> You are relatively unconcerned with both <span class='diff'>achieving success</span> and taking pleasure in life. <span class='diff'>You prefer activities with a purpose greater than just personal enjoyment. And you make decisions with little regard for how they show off your talents.</span> </td> </tr> </table> <p> An old friend, who I've known since university, is a clinical psychologist with a PhD and works as a doctor in a mental hospital. His comments on the results were:</p> <blockquote> <p> I think that some aspects of the profile are pretty accurate; however, other aspects such as calling you shrewd and skeptical are a bit evaluate: I would instead say intelligent and not naive. The openness results would seem pretty accurate. </p> <p> I think you're probably a bit more extraverted than this analysis suggests. </p> <p> The ‘emotional range’ factor of the Big 5 is typically labeled Neuroticism. I kind of think of Agreeableness as a compliance/ conformity dimension, and I if I am not mistaken, it is not unusual for (male) entrepreneurs to score low on that dimension. </p> </blockquote> <h2>But Wait, There's More!</h2> <p> IBM's Personality Insights provides additional information that explains the above in more detail: </p> <table class="table table_striped table_cell_top table_cell_vspace table_cell_justify"> <tr> <th width="50%">Birthday Posting Summary</th> <th width="50%"><a href="/blog/2008/04/28/cult-of-software-god.html">Humorous Posting</a> Summary</th> </tr> <tr> <td> You are <b>likely</b> to: <ul> <li> <span class='diff'>like musical movies</span><br /> <span class="comment">Oops, that was way off!</span> </li> <li> be sensitive to ownership cost when buying automobiles </li> <li> have experience playing music <br /> <span class="comment">True: I am a multi-instrumentalist</span> </li> </ul> <td> You are <b>likely</b> to: <ul> <li> <span class='diff'>like historical movies</span> <br /> <span class="comment">True!</span></li> <li>be sensitive to ownership cost when buying automobiles </li> <li> have experience playing music</li> </ul> </tr> <tr> <td> You are <b>unlikely</b> to: <ul> <li> be influenced by social media during product purchases <span class="comment">The truth is more complex</span> </li> <li> prefer style when buying clothes <span class="comment">True, and to compensate I try to shop for clothes with carefully selected friends</span> </li> <li><span class='diff'>like country music</span> <span class="comment">Spot on!</span></li> </ul> </td> <td> <p style="bold"> You are <b>unlikely</b> to: </p> <ul> <li> be influenced by social media during product purchases <br> <br> </li> <li> prefer style when buying clothes <br> <br> <br> </li> <li> <span class='diff'>be influenced by brand name when making product purchases</span> </li> </ul> </td> </tr> </table> <p> I think that the above was a pretty good assessment of me, and the detailed breakdowns which follow are interesting. Before you look at that, however, you should know that the <a href="https://www.verywellmind.com/the-big-five-personality-dimensions-2795422" rel="nofollow">Big 5 Personality Model</a> defines the 5 traits using words with meanings that might seem different from the meanings you might expect. The 5 traits are: <a href="https://console.bluemix.net/docs/services/personality-insights/openness.html" rel="nofollow"><i>openness</i></a>, <a href="https://console.bluemix.net/docs/services/personality-insights/conscientiousness.html" rel="nofollow"><i>conscientiousness</i></a>, <a href="https://console.bluemix.net/docs/services/personality-insights/extroversion.html" rel="nofollow"><i>extroversion</i></a>, <a href="https://console.bluemix.net/docs/services/personality-insights/agreeableness.html" rel="nofollow"><i>agreeableness</i></a> and <a href="https://console.bluemix.net/docs/services/personality-insights/emotional-range.html" rel="nofollow"><i>emotional range</i></a>. </p> <h2 id="pnv">Personality, Needs and Values</h2> <p> Each of the 5 traits are broken down into various aspects. For example, openness consists of <i>adventurousness</i>, <i>artistic interests</i>, <i>emotionality</i>, <i>imagination</i>, <i>intellect</i>, and <i>authority-challenging</i>. Again, these terms are <a href="https://console.bluemix.net/docs/services/personality-insights/openness.html#dimensions" rel="nofollow">defined in specific ways</a> that might be different from the definitions that you might expect. </p> <table class="table table_striped table_cell_top table_cell_vspace table_cell_justify"> <tr> <th width="50%">Birthday Posting Summary</th> <th width="50%"><a href="/blog/2008/04/28/cult-of-software-god.html">Humorous Posting</a> Summary</th> </tr> <tr> <td> <p><br /><br /> <i>1724 words; decent analysis.</i> </p> <div style=""> <picture> <source srcset="/blog/images/ibmPersonalityInsightMslinnSliders.webp" type="image/webp"> <source srcset="/blog/images/ibmPersonalityInsightMslinnSliders.png" type="image/png"> <img src="/blog/images/ibmPersonalityInsightMslinnSliders.png" title="Birthday posting summary by IBM Personality Insights" class=" rounded shadow zoom" alt="Birthday posting summary by IBM Personality Insights" /> </picture> </div> </td> <td> <p> <i>516 words; we need a minimum of 600, preferably 1,200 or more, to compute statistically significant estimates.</i> </p> <div style=""> <picture> <source srcset="/blog/images/ibmPersonalityInsightMslinnSliders2.webp" type="image/webp"> <source srcset="/blog/images/ibmPersonalityInsightMslinnSliders2.png" type="image/png"> <img src="/blog/images/ibmPersonalityInsightMslinnSliders2.png" title="Humorous Posting summary by IBM Personality Insights" class=" rounded shadow zoom" alt="Humorous Posting summary by IBM Personality Insights" /> </picture> </div> </td> </tr> </table> <h2 id="detail">Detailed Breakdown</h2> <p> Here is my detailed breakdown, shown as sunburst charts: </p> <table class="table table_striped table_cell_top table_cell_vspace table_cell_justify" style="width: 100%"> <tr> <th width="50%"> Birthday Posting Summary </th> <th width="50%"> <a href="/blog/2008/04/28/cult-of-software-god.html">Humorous Posting</a> Summary </th> </tr> <tr> <td> <div style=""> <picture> <source srcset="/blog/images/ibmPersonalityInsightMslinnSunburst.webp" type="image/webp"> <source srcset="/blog/images/ibmPersonalityInsightMslinnSunburst.png" type="image/png"> <img src="/blog/images/ibmPersonalityInsightMslinnSunburst.png" title="Birthday posting summary by IBM Personality Insights" class=" rounded shadow zoom" alt="Birthday posting summary by IBM Personality Insights" /> </picture> </div> </td> <td> <div style=""> <picture> <source srcset="/blog/images/ibmPersonalityInsightMslinnSunburst2.webp" type="image/webp"> <source srcset="/blog/images/ibmPersonalityInsightMslinnSunburst2.png" type="image/png"> <img src="/blog/images/ibmPersonalityInsightMslinnSunburst2.png" title="Humorous posting summary by IBM Personality Insights" class=" rounded shadow zoom" alt="Humorous posting summary by IBM Personality Insights" /> </picture> </div> </td> </tr> </table> <p> The sunburst charts paint me as an unusual person. If this personality assessment is accurate, I am rather complex. However, further reading suggests that people with very high openness scores defy most structured evaluations, and I score in the 99<sup>th</sup> percentile for openness. </p> <table class="table table_striped table_cell_top table_cell_vspace table_cell_justify" style="width: 100%"> <tr> <th width="50%"> Birthday Posting Summary </th> <th width="50%"> <a href="/blog/2008/04/28/cult-of-software-god.html">Humorous Posting</a> Summary </th> </tr> <tr> <td> <b>Openness to experience</b>: high (99<sup>th</sup> percentile). This means I have an unusually high fluid intelligence (I am able to learn complex concepts and tasks very quickly), and I am likely to be eccentric. <br /> <span class="comment">I believe this to be true</span>. </td> <td> <b>Openness to experience</b>: high (99<sup>th</sup> percentile) &ndash; <span class="comment">Same</span> </td> </tr> <tr> <td> <b>Extraversion</b>: Low (15th percentile) &ndash; Cold, withdrawn, unfriendly. <span class="comment">That feels harsh.</span> </td> <td> <b>Extraversion</b>: Even lower (6<sup>th</sup> percentile). <span class="comment">Yikes!</span> </td> </tr> <tr> <td> <b>Agreeableness</b>: Low (5<sup>th</sup> percentile) &ndash; Independent, tough, dominant, possibly manipulative. <span class="comment">Yes, I make up my mind and I follow what I believe to be the appropriate course, regardless of what others might say or do.</span> <br /> Additionally, I have very strong sympathy (97<sup>th</sup> percentile) and I am rather uncompromising (82nd percentile), strongly cooperative (82<sup>nd</sup> percentile) with strong altruism (87<sup>th</sup> percentile). <br /> <span class="comment">Perhaps that means that I am a crusty individual with a warm heart.</span> <br /> According to the breakdown, I am quite trusting (82<sup>nd</sup> percentile), yet I am also very cautious (90<sup>th</sup> percentile), so for me I go with &ldquo;trust but verify&rdquo;. </td> <td> <b><a href="https://cloud.ibm.com/docs/services/personality-insights?topic=personality-insights-agreeableness" rel="nofollow">Agreeableness</a></b>: Even lower (0<sup>th</sup> percentile). <span class="comment">Yikes!</span> </td> </tr> <tr> <td> <b>Emotional range</b>: average (> 59<sup>th</sup> percentile). <span class="comment"><a href="https://cloud.ibm.com/docs/services/personality-insights?topic=personality-insights-emotionalRange" rel="nofollow">IBM defines this</a> as &ldquo;the extent to which a person's emotions are sensitive to the individual's environment&rdquo;</span> </td> <td> <b>Emotional range</b>: very high (> 91<sup>st</sup> percentile). When coupled with low agreeableness, IBM predicts: temperamental, irritable, quarrelsome, impatient, grumpy. When coupled with low conscientiousness, IBM predicts: compulsive, nosy, self-indulgent, forgetful, impulsive. When coupled with low extroversion, IBM predicts: guarded, fretful, insecure, pessimistic, secretive. When coupled with high openness, IBM predicts: excitable, passionate, sensual. <span class="comment">Hmm, quarrelsome, impatient, compulsive, self-indulgent, forgetful, insecure, pessimistic and yet passionate and sensual. What a combination!</span> </td> </tr> <tr> <td> <b>Low Conservation (1<sup>st</sup> percentile)</b> &ndash; <span class="comment">This is called Hedonism in the other graph.</span> </td> <td> <b>Hedonism</b>: very low (1<sup>st</sup> percentile) &ndash; <span class="comment">This is called Conservation in the other graph.</span> Within this category I scored low self-enhancement, low Hedonism, low Openness to change and low conservation. </td> </tr> <tr> <td> <b>Low Harmony (5<sup>th</sup> percentile)</b> &ndash; <span class="comment">This is called Closeness in the other graph.</span> </td> <td> <b>Closeness:</b> low (1<sup>th</sup> percentile) &ndash; <span class="comment">This is called Harmony in the other graph.</span> Within this category I scored low Excitement, Harmony, Ideal, Liberty, Love, Self-expression and Stability. I also scored high Curiosity (78<sub>th</sub> percentile) and high Structure (88<sup>th</sup> percentile). <span class="comment">Sounds like I'm pretty much a curious robot.</span> </td> </tr> </table> <p> These traits, if true, would make me a good expert witness, and a good evaluator for technical due diligence. This might also explain my propensity for constantly inventing new things. </p> <p> According to these results, I also registered some extreme <a href="https://console.bluemix.net/docs/services/personality-insights/needs.html#needs" rel="nofollow">needs</a> and <a href="https://console.bluemix.net/docs/services/personality-insights/values.html#values" rel="nofollow">values</a>. Because the results of analyzing both documents were almost the same, I show them together. </p> <p> First, let's look at the results that I identify with: </p> <table class="table table_striped table_cell_top table_cell_vspace table_cell_justify" style="width: 100%"> <tr> <th> Both articles </th> </tr> <tr> <td> An extremely liberal mindset (<b>conservation</b>: 1st percentile) </td> </tr> <tr> <td> Virtually no interest in comfort (<b>hedonism</b>: 2nd percentile) </td> </tr> <tr> <td> Almost no interest in social power, authority, wealth, success, capability, etc. (<a href="https://www.researchgate.net/publication/322627891_The_five_pillars_of_self-enhancement_and_self-protection" rel="nofollow"> <b>self-enhancement</b></a>: 4th percentile) </td> </tr> <tr> <td> Almost no interest in <b>harmony</b> (5th percentile) </td> </tr> <tr> <td> Strong <b>curiosity</b> (87th percentile) </td> </tr> <tr> <td> Low connection to family and setting up a home (<b>closeness</b>: 9%) </td> </tr> <tr> <td> Very little respect, commitment, and acceptance of the customs and ideas that culture and/or religion provides. </td> </tr> </table> <p> I think the following needs assessments are way off the mark. Perhaps I protest too much? </p> <table class="table table_striped table_cell_top table_cell_vspace table_cell_justify" style="width: 100%"> <tr> <td> Very little interest in just having fun for its own sake (<b>excitement</b>: 7th percentile). </td> </tr> <tr> <td> Low desire for perfection or a sense of community <span class="comment">The truth is complicated: I have spent decades building professional and musical communities, yet I am isolated in many ways</span> </td> </tr> <tr> <td> Low interest in discovering and asserting their identities (<b>self-expression</b>: 9%) <span class="comment">This seems way off the mark, for example consider the motivation for my spending thousands of hours on EmpathyWorks.</span> </td> </tr> </table> Evaluating Blockchain Companies 2018-08-29T00:00:00-04:00 https://mslinn.github.io/blog/2018/08/29/evaluating-blockchain-companies <div style=""> <picture> <source srcset="/blog/images/2018-10-19_10-51-13_690x511.webp" type="image/webp"> <source srcset="/blog/images/2018-10-19_10-51-13_690x511.png" type="image/png"> <img src="/blog/images/2018-10-19_10-51-13_690x511.png" title="Mike Slinn presents" class=" liImg " alt="Mike Slinn presents" /> </picture> </div> <div> <p> I have performed <a href="/evaluation/index.html">technical due diligence</a> for investors since the mid-1980s. In this 40-minute presentation I explain how I evaluate blockchain-related technology companies. </p> <p> This talk was presented at the 6th Annual Global Big Data Conference in Santa Clara, California on August 29, 2018. The promotional video recording is <a href="https://www.youtube.com/watch?v=o_9q2USRzzI" target="_blank" rel="nofollow">here</a>. </p> <p> <a href="https://www.infoq.com/presentations/evaluate-blockchain-companies/" target="_blank" rel="nofollow">InfoQ produced the presentation</a> in their unique format. </p> <p> The slides are <a href="https://www.slideshare.net/mslinn/evaluating-blockchain-companies/mslinn/evaluating-blockchain-companies" target="_blank" rel="nofollow">here</a>. </p> <p> The video recording is <a href="https://big.mslinn.com/video/18-aug-evaluatingbccompanies.mp4" target="_blank" rel="nofollow">here</a>. </p> </div> <div style="text-align: center;"> <picture> <source srcset="/blog/images/6thGlobalBlockchain.webp" type="image/webp"> <source srcset="/blog/images/6thGlobalBlockchain.png" type="image/png"> <img src="/blog/images/6thGlobalBlockchain.png" title="Mike Slinn presents" class="center liImg2 rounded shadow" alt="Mike Slinn presents" /> </picture> </div> Bob Summerwill 2018-08-28T00:00:00-04:00 https://mslinn.github.io/blog/2018/08/28/summerwill <p> This is a great photo of Bob Summerwill and me in San Francisco August 2018! </p> <div style=""> <picture> <source srcset="/assets/images/ethereum/meBobSummerwill_690x518.webp" type="image/webp"> <source srcset="/assets/images/ethereum/meBobSummerwill_690x518.png" type="image/png"> <img src="/assets/images/ethereum/meBobSummerwill_690x518.png" title="Mike Slinn and Bob Summerwill in San Francisco August 2018" class=" liImg2 rounded shadow" alt="Mike Slinn and Bob Summerwill in San Francisco August 2018" /> </picture> </div> Keynote Panel Discussion - The Future of Blockchain 2018-08-23T00:00:00-04:00 https://mslinn.github.io/blog/2018/08/23/blockchain-conference-2018-08-28 <div style="text-align: right;"> <picture> <source srcset="/blog/images/mikeBigData_500x431.webp" type="image/webp"> <source srcset="/blog/images/mikeBigData_500x431.png" type="image/png"> <img src="/blog/images/mikeBigData_500x431.png" title="Global Big Data Conference featuring Mike Slinn as a speaker" class="right liImg2 rounded shadow" style="width: 100%; height: auto" alt="Global Big Data Conference featuring Mike Slinn as a speaker" /> </picture> </div> <p> I will participate in a keynote panel discussion on the Future of Blockchain on August 23 from 4:50 PM to 6:00 PM at the <a href="https://sanjose.eventful.com/events/blockchain-new-infrastructure-ai-/E0-001-112130527-9" target="_blank" rel="nofollow">6th Annual Global Big Data Conference</a> in Santa Clara, California. </p> <p> Following are some ideas I hope to discuss with the other members of the panel. First, I would like to remind the reader that blockchain data is passive; without a program to access it blockchain data is inert. Now I'd like to give the definition that I will use for this discussion. </p> <p> The word <i>blockchain</i> is defined by <a href="https://en.wikipedia.org/wiki/Blockchain" target="_blank" rel='nofollow'>Wikipedia</a> as: &ldquo;a growing list of records, called blocks, which are linked using cryptography.&rdquo; The main theme I&rsquo;d like to personally bring forward during this discussion is that Blockchain does not imply or require decentralization, or even distributed systems. Instead, blockchain is a useful technology in its own right, even if it is not distributed / decentralized. I have nothing against decentralization, when used appropriately; the point I want to make is that decentralization is not necessary for blockchain to provide value. </p> <p> Here are the use cases I want to talk about: </p> <h2 id="dbs">#1: Secure Mobile and Embedded Databases</h2> <p> For me, <i>blockchain</i> is a file format that yields immutable data, highly resistant to modification. It is not a database, instead, it is a generic type of storage technology. Yes, an API could be provided that provides a structured interface to blockchain data, but there is no universally accepted standard in use for this in any blockchain implementation that I am aware of. Such a standard should exist, and I would be interested in learning about any standards work in this area. </p> <p> A database built using blockchain would not support the full <a href="https://en.wikipedia.org/wiki/Create,_read,_update_and_delete" target="_blank" rel='nofollow'>CRUD</a> API (create, read, update, and delete) functionality traditionally supported by SQL. Instead, only create and read functionality would be supported. </p> <p> This seems especially useful for mobile and embedded data recorders, for example flight recorders used in airplanes. I have not yet attempted to compute how feasible it might be to use blockchain to store video or audio streams from devices such as <a href="https://www.amazon.com/slp/police-body-camera/4r72vj5jucggbyv" target="_blank" rel='nofollow'>police body cameras</a>. </p> <h2 id="data">#2: Secure Data Distribution</h2> <p> Blockchain is a data storage technology that is known to be highly resistant to corruption or modification. Because data can only be created or read, but not modified or deleted, if you have a file of blockchain data that passes inspection you can be confident that you have all the data that was available at the time the file was last updated. </p> <p> Blockchain data could be stored in RFID tags. This is possible because RFID tags exist which can <a href="https://www.rfidjournal.com/faq/show?66" target="_blank" rel='nofollow'>store up to 66 KB of data</a>, and read/write RFID tags are available. Read-write RFID tags usually have a serial number that can&rsquo;t be written over. Additional blocks of data can be used to store additional information about the items the tag is attached to (these can usually be locked to prevent overwriting of data). </p> <p> This would allow blockchain to be used for <a href="https://www.theglobalist.com/smart-passports-making-travel-safer/" target="_blank" rel='nofollow'>smart passports</a> and smart ID cards. Secure, smart passports would provide extra security for passport/ID cardholders and would make counterfeiting extremely difficult, and would provide a digital record of countries visited, instead of relying on passport stamps, which are easy to counterfeit. </p> <p> Similarly, smart ID cards could hold mutable yet secure data. </p> <h2 id="builders">#3: Secure Software Build Systems</h2> <p> There have been many instances of corporate spies infiltrating software companies or their repositories. The perpetrators altered the source code, accompanying data or modified other build dependencies. When the next version of the software is published, all the customers who install updates will potentially be compromised. </p> <p> <a href="https://en.wikipedia.org/wiki/Git" target="_blank" rel='nofollow'><code>git</code></a> uses similar cryptography techniques as does blockchain. A blockchain-based build system would use a git tag to kick off build, and would need to also store and compare hashes for all dependencies to ensure that they had not changed. Because the software build toolchain is also a dependency, hashes for all the software used in the toolchain would need to be validated at build time. </p> <h2 id="dist">#4: Secure Software Distribution</h2> <p> Computer viruses have been a problem for decades. There have been many instances where programs have viruses embedded after they are published. This is generally true for any software downloaded from a torrent site. If software was packaged using blockchain technology, it could be retrieved for installation with confidence. Software installers that used blockchain in this way could be trusted. </p> <hr /> <h2 id="panel">The Panelists</h2> <p> After the panel, we panelists were photographed together. From left to right: Hayden Kirkpatrick, VP of Strategy at Esurance (moderator); Karen Hsu, CRO at BlockXypher; Steve Beauregard, CRO at Bloq; Mike Slinn, CTO at Micronautics Research (holding the microphone); Mark Javier, Account Executive at Ambisafe; and Camille Sanandaji, CEO at Foodstems. </p> <div style=""> <picture> <source srcset="/assets/images/ethereum/futureOfBlockchainPanelCrop2_690x192.webp" type="image/webp"> <source srcset="/assets/images/ethereum/futureOfBlockchainPanelCrop2_690x192.png" type="image/png"> <img src="/assets/images/ethereum/futureOfBlockchainPanelCrop2_690x192.png" title="Hayden Kirkpatrick (moderator), Karen Hsu, Steve Beauregard, Mike Slinn, and Camille Sanandaji" class=" liImg2 rounded shadow" alt="Hayden Kirkpatrick (moderator), Karen Hsu, Steve Beauregard, Mike Slinn, and Camille Sanandaji" /> </picture> </div> <div style=""> <picture> <source srcset="/assets/images/ethereum/blockchainFuturePanel_690x690.webp" type="image/webp"> <source srcset="/assets/images/ethereum/blockchainFuturePanel_690x690.png" type="image/png"> <img src="/assets/images/ethereum/blockchainFuturePanel_690x690.png" title="Mark Javier, Mike Slinn, Hayden Kirkpatrick (moderator), Steve Beauregard, Karen Hsu, and Camille Sanandaji" class=" liImg2 rounded shadow" alt="Mark Javier, Mike Slinn, Hayden Kirkpatrick (moderator), Steve Beauregard, Karen Hsu, and Camille Sanandaji" /> </picture> </div> Windows Subsystem for Linux Revisited 2018-08-20T00:00:00-04:00 https://mslinn.github.io/blog/2018/08/20/wsl-revisited <div style=""> <picture> <source srcset="/blog/images/wsl1_690x431.webp" type="image/webp"> <source srcset="/blog/images/wsl1_690x431.png" type="image/png"> <img src="/blog/images/wsl1_690x431.png" title="Windows Subsystem for Linux (WSL)" class=" liImg2 rounded shadow" alt="Windows Subsystem for Linux (WSL)" /> </picture> </div> <p> I&rsquo;ve been using <a href="https://docs.microsoft.com/en-us/windows/wsl/install-win10" target="_blank" rel='nofollow'>Windows Subsystem for Linux</a> (WSL) headless since it was first released with Windows 10 version 1607 in August 2016. The <a href="https://blogs.msdn.microsoft.com/commandline/2018/03/07/windows10v1803/" target="_blank" rel='nofollow'>April 2018 release</a> of Windows 10 (version 1803) significantly improved WSL. </p> <p> It is possible to work with Ubuntu graphically on a vanilla Windows machine. No special drivers are required. No special Linux or Ubuntu support is required from the computer vendor. </p> <p> This is my setup for running an X client like <a href="https://www.x.org/releases/X11R7.5/doc/man/man1/xeyes.1.html" target="_blank" rel='nofollow'><code>xeyes</code></a> or <a href="https://www.jetbrains.com/idea/" target="_blank" rel='nofollow'>IntelliJ IDEA</a> from WSL or WSL2, accessed via a Windows X server like <a href="https://sourceforge.net/projects/vcxsrv/" target="_blank" rel='nofollow'>VcXsrv</a>. These scripts assume <a href="https://www.microsoft.com/en-us/p/ubuntu-1804/9n9tngvndl3q" target="_blank" rel='nofollow'>Ubuntu 18.04 was installed</a> under WSL. </p> <div style=""> <picture> <source srcset="/blog/images/wsl.webp" type="image/webp"> <source srcset="/blog/images/wsl.png" type="image/png"> <img src="/blog/images/wsl.png" title="Desktop showing Windows Subsystem for Linux (WSL)" class=" liImg2 rounded shadow" alt="Desktop showing Windows Subsystem for Linux (WSL)" /> </picture> </div> <h2 id="essential">Essential Scripts</h2> <p>Here are bash scripts to install everything:</p> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=wsl_init"> </script> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=wsl_setup"> </script> <h2 id="optional">Optional Scripts</h2> <p>The remaining scripts are all optional.</p> <h3 id="vnc">VNC</h3> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=wsl_vnc"> </script> <p>These bash scripts allow VNC to connect to a remote machine or to WSL on the local machine.</p> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=vncRun"> </script> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=wsl"> </script> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=vncSample"> </script> <h2 id="packages">Packages and Programming Environments</h2> <p>Installation of various packages and programming environments follow.</p> <h3 id="git">Git</h3> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=wsl_git"> </script> <h3 id="python">Python</h3> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=wsl_python"> </script> <h3 id="nodejs">NodeJS</h3> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=wsl_node"> </script> <h3 id="jvm">Java Virtual Machine</h3> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=wsl_jvm"> </script> <h3 id="misc">Miscellaneous</h3> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=wsl_misc"> </script> <h3 id="atom">Atom Editor</h3> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/7be077ebeb9c172572e211e354a8c5f8.js?file=wsl_atom"> </script> <h3 id="wine">Wine</h3> <p> The advent of WSL means that Wine is no longer required! </p> <h2 id="update">Update Aug 17, 2020</h2> <p> Here are my notes on upgrading WSL and WSL2 from Ubuntu 19.11 to Ubuntu 20.04. </p> <noscript><pre>400: Invalid request</pre></noscript><script src="https://gist.github.com/50a3a8d26cbab51c634ed8e2edf129ae.js"> </script> Ethereum Source Code Walkthrough 2018-06-13T00:00:00-04:00 https://mslinn.github.io/blog/2018/06/13/evm-source-walkthrough <div class="formalNotice rounded shadow" id="about"> <h2>April 27, 2020</h2> <p> <div style=""> <picture> <source srcset="/assets/images/ethereum/Ethereum_logo_2014.svg" type="image/webp"> <source srcset="/assets/images/ethereum/Ethereum_logo_2014.svg" type="image/png"> <img src="/assets/images/ethereum/Ethereum_logo_2014.svg" title="Ethereum logo" class=" liImg right" style="height: 100px;" alt="Ethereum logo" /> </picture> </div> This was a sample work in progress as part of a proposal to the Ethereum Foundation Grants committee. They declined to fund this activity, but gave no reason and no feedback as to what might be acceptable. The preparation of my proposal took weeks, and the preliminary feedback that I had received from many knowledgeable people was that it had great value. </p> <p> I felt that the evaluation process was broken, in fact the entire organization was broken, and there was little hope that Ethereum would ever become a professional organization. 2 years later, I believe history has proved me right. This was the last blockchain-related initiative I participated in. </p> <p> I no longer maintain <code><a href="/blog/2017/11/29/web3j-scala.html" target="_blank" rel='nofollow'>web3j-scala</a></code>, an Ethereum-related project for Scala programmers. I created that open source project and worked on it for free for 3 years. It has been forked and I&rsquo;ve been told it is or was used in production. However, I see no reason to continue working for free on it. Others can carry the project forward if they want. </p> </div> <p> This is a brief walkthrough of some of the core source files for smart contracts in the official <a href="https://golang.org/" target="_blank" rel='nofollow'>Go language</a> Ethereum implementation, which includes the <a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//core/vm" target="_blank" rel='nofollow'><code>geth</code></a> command-line Ethereum client program, along with many other programs. Ethereum clients include an implementation of the Ethereum Virtual Machine (EVM), which are able to parse and verify the Ethereum blockchain, including smart contracts, and provides interfaces to create transactions and mine blocks. </p> <p> I&rsquo;ve added some suggestions for how the source code might be improved. If there is general agreement that these suggestions make sense (tell me in the comments!) then I&rsquo;ll create a pull request. </p> <h2>License</h2> <p> <div style=""> <picture> <source srcset="/assets/images/lgplv3-147x51.webp" type="image/webp"> <source srcset="/assets/images/lgplv3-147x51.png" type="image/png"> <img src="/assets/images/lgplv3-147x51.png" title="LGPL logo" class=" right" alt="LGPL logo" /> </picture> </div> This Ethereum client project was released under the <a href="https://www.gnu.org/licenses/lgpl-3.0.en.html" target="_blank" rel='nofollow'>GNU Lesser General Public License, version 3</a> or later, which permits use of the code as a library in proprietary programs. </p> <h2>Source Files</h2> <p> The <a href="https://github.com/hhatto/gocloc" target="_blank" rel='nofollow'><code>gocloc</code></a> program counted the following source files and lines: </p> <table class="table table_striped table_right"> <tr> <th>Language</th> <th>Files</th> <th>Blank Lines</th> <th>Comment Lines</th> <th>Code Lines</th> </tr> <tr> <th>Go</th> <td>1824</td> <td>58,134</td> <td>81,861</td> <td>639,435</td> </tr> <tr> <th>C</th> <td>55</td> <td>17,257</td> <td>30,909</td> <td>84,719</td> </tr> <tr> <th>C Header</th> <td>97</td> <td>2,559</td> <td>6,318</td> <td>15,083</td> </tr> <tr> <th>Markdown</th> <td>88</td> <td>3,152</td> <td>0</td> <td>9,175</td> </tr> <tr> <th>JavaScript</th> <td>13</td> <td>1,845</td> <td>4,495</td> <td>7,986</td> </tr> <tr> <th>Assembly</th> <td>39</td> <td>557</td> <td>957</td> <td>3,783</td> </tr> <tr> <th>JSON</th> <td>17</td> <td>0</td> <td>0</td> <td>2,065</td> </tr> <tr> <th>Protocol Buffers</th> <td>2</td> <td>113</td> <td>40</td> <td>1,030</td> </tr> <tr> <th>Plain Text</th> <td>11</td> <td>217</td> <td>0</td> <td>954</td> </tr> <tr> <th>C++</th> <td>4</td> <td>132</td> <td>102</td> <td>937</td> </tr> <tr> <th>BASH</th> <td>10</td> <td>178</td> <td>315</td> <td>931</td> </tr> <tr> <th>Perl</th> <td>10</td> <td>268</td> <td>1,289</td> <td>879</td> </tr> <tr> <th>JSX</th> <td>11</td> <td>119</td> <td>245</td> <td>722</td> </tr> <tr> <th>XML</th> <td>9</td> <td>0</td> <td>0</td> <td>651</td> </tr> <tr> <th>M4</th> <td>4</td> <td>79</td> <td>99</td> <td>649</td> </tr> <tr> <th>YAML</th> <td>20</td> <td>77</td> <td>42</td> <td>581</td> </tr> <tr> <th>NSIS</th> <td>5</td> <td>86</td> <td>154</td> <td>446</td> </tr> <tr> <th>Java</th> <td>4</td> <td>143</td> <td>187</td> <td>438</td> </tr> <tr> <th>Makefile</th> <td>11</td> <td>101</td> <td>84</td> <td>381</td> </tr> <tr> <th>Python</th> <td>6</td> <td>154</td> <td>250</td> <td>339</td> </tr> <tr> <th>HTML</th> <td>3</td> <td>15</td> <td>9</td> <td>245</td> </tr> <tr> <th>Solidity</th> <td>7</td> <td>56</td> <td>171</td> <td>213</td> </tr> <tr> <th>Bourne Shell</th> <td>6</td> <td>23</td> <td>25</td> <td>119</td> </tr> <tr> <th>CMake</th> <td>1</td> <td>9</td> <td>0</td> <td>35</td> </tr> <tr> <th>Awk</th> <td>1</td> <td>4</td> <td>4</td> <td>17</td> </tr> <tr> <th>TOML</th> <td>1</td> <td>0</td> <td>0</td> <td>3</td> </tr> <tr> <th>Total</th> <th class="table_right">2,260</th> <th class="table_right">85,278</th> <th class="table_right">127,556</th> <th class="table_right">771,825</th> </tr> </table> <h2>Packages</h2> <p> I used the following incantation to discover that <code>geth</code> defines 244 packages: </p> <pre data-lt-active='false'><span class="unselectable">$ </span>grep -rh "^package" | grep -v "not installed" | \ tr -d ';' | sed 's^//.*^^' | awk '{$1=$1};1' | \ sort | uniq | wc -l</pre> <p> I won't list them all. The <a href="https://godoc.org/github.com/ethereum/go-ethereum#pkg-subdirectories" target="_blank" rel='nofollow'><code>godoc</code></a> for the project contains much of the <a href="https://godoc.org/github.com/ethereum/go-ethereum#pkg-subdirectories" target="_blank" rel='nofollow'>following documentation</a> for the top-level packages. I provided the rest of the information from disparate sources, including reading the source code: </p> <p> </p> <table class="table table_striped"> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//accounts" target="_blank" rel='nofollow'>accounts</a></th> <td>implements high-level Ethereum account management.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/blob/master/trie/trie.go" target="_blank" rel='nofollow'>trie</a></th> <td>provides a binary Merkle tree implementation.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd" target="_blank" rel='nofollow'>cmd</a></th> <td>Contains the following command-line tools. Most tools support the <code>--help</code> option. <table class="table table_striped" style="margin-top: 0.5em"> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd/abigen" target="_blank" rel='nofollow'>abigen</a></th> <td>source code generator to convert Ethereum contract definitions into easy to use, compile-time type-safe Go packages. It operates on plain <a href="https://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI" target="_blank" rel='nofollow'>Ethereum contract ABIs</a> with expanded functionality if the contract bytecode is also available. However it also accepts Solidity source files, making development much more streamlined. Please see the <a href="https://github.com/ethereum/go-ethereum/wiki/Native-DApps:-Go-bindings-to-Ethereum-contracts" target="_blank" rel='nofollow'>Native DApps wiki page</a> for details.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd/bootnode" target="_blank" rel='nofollow'>bootnode</a></th> <td>runs a bootstrap node for the Ethereum Discovery Protocol. This is a stripped-down version of geth that only takes part in the network node discovery protocol, and does not run any of the higher level application protocols. It can be used as a lightweight bootstrap node to aid in finding peers in private networks.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd/clef" target="_blank" rel='nofollow'>clef</a></th> <td>a standalone signer that manages keys across multiple Ethereum-aware apps such as Geth, Metamask, and cpp-ethereum. <i>Alpha quality, not released yet.</i></td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd/ethkey" target="_blank" rel='nofollow'>ethkey</a></th> <td>a key/wallet management tool for Ethereum keys. Allows user to add, remove and change their keys, and supports cold wallet device-friendly transaction inspection and signing. <a href="https://github.com/ethereum/guide/blob/master/ethkey.md" target="_blank" rel='nofollow'>This documentation</a> was written for the C++ Ethereum client implementation, but it is probably suitable for the Go implementation as well. </td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd/evm" target="_blank" rel='nofollow'>evm</a></th> <td>a version of the EVM (Ethereum Virtual Machine) for running bytecode snippets within a configurable environment and execution mode. Allows isolated, fine-grained debugging of EVM opcodes. Example usage: <pre data-lt-active='false'>evm --code 60ff60ff --debug</pre></td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd/faucet" target="_blank" rel='nofollow'>faucet</a></th> <td>an Ether faucet backed by a light client.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd/geth" target="_blank" rel='nofollow'>geth</a></th> <td>official command-line client for Ethereum. It provides the entry point into the Ethereum network (main-, test- or private net), capable of running as a full node (default) archive node (retaining all historical state) or a light node (retrieving data live). It can be used by other processes as a gateway into the Ethereum network via JSON RPC endpoints exposed on top of HTTP, WebSocket and/or IPC transports. For more information see the <a href="https://github.com/ethereum/go-ethereum/wiki/Command-Line-Options" target="_blank" rel='nofollow'>CLI Wiki page</a>.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd/p2psim" target="_blank" rel='nofollow'>p2psim</a></th> <td>a simulation HTTP API. <a href="https://godoc.org/github.com/ethereum/go-ethereum/cmd/p2psim" target="_blank" rel='nofollow'>Docs are here</a>.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd/puppeth" target="_blank" rel='nofollow'>puppeth</a></th> <td>assembles and maintains private networks.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd/rlpdump" target="_blank" rel='nofollow'>rlpdump</a></th> <td>a pretty-printer for RLP data. RLP (Recursive Length Prefix) is the data encoding used by the Ethereum protocol. Sample usage: <pre data-lt-active='false'>rlpdump --hex CE0183FFFFFFC4C304050583616263</pre></td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/blob/master/swarm/README.md" target="_blank" rel='nofollow'>swarm</a></th> <td>provides the <code>bzzhash</code> command, which computes a swarm tree hash, and implements the swarm daemon and tools. See <a href="https://swarm-guide.readthedocs.io"> the swarm documentation</a> for more information.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//cmd/wnode" target="_blank" rel='nofollow'>wnode</a></th> <td>simple Whisper node. It could be used as a stand-alone bootstrap node. Also could be used for different test and diagnostics purposes.</td> </tr> </table> </td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//common" target="_blank" rel='nofollow'>common</a></th> <td>contains various helper functions worth checking out</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//consensus" target="_blank" rel='nofollow'>consensus</a></th> <td>implements different Ethereum consensus engines (which must conform to the <a href="https://godoc.org/github.com/ethereum/go-ethereum/consensus#Engine" target="_blank" rel='nofollow'><code>Engine</code> interface</a>): <a href="https://godoc.org/github.com/ethereum/go-ethereum/consensus/clique" target="_blank" rel='nofollow'><code>clique</code></a> implements proof-of-authority consensus, and <a href="https://godoc.org/github.com/ethereum/go-ethereum/consensus/ethash" target="_blank" rel='nofollow'>ethash</a> implements proof-of-work consensus.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//console" target="_blank" rel='nofollow'>console</a></th> <td> Ethereum implements a JavaScript runtime environment (JSRE) that can be used in either interactive (console) or non-interactive (script) mode. Ethereum's JavaScript console exposes the full web3 JavaScript Dapp API and the admin API. <a href="https://github.com/ethereum/go-ethereum/wiki/JavaScript-Console" target="_blank" rel='nofollow'>More documentation is here.</a> This package implements JSRE for the <code>geth console</code> and <code>geth console</code> subcommands. </td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8/containers" target="_blank" rel='nofollow'>containers</a></th> <td></td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//contracts" target="_blank" rel='nofollow'>contracts</a></th> <td></td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//core" target="_blank" rel='nofollow'>core</a></th> <td>implements the Ethereum consensus protocol, implements the Ethereum Virtual Machine, and other miscellaneous important bits</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//crypto" target="_blank" rel='nofollow'>crypto</a></th> <td>cryptographic implementations</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//dashboard" target="_blank" rel='nofollow'>dashboard</a></th> <td></td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//eth" target="_blank" rel='nofollow'>eth</a></th> <td>implements the Ethereum protocol</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//ethclient" target="_blank" rel='nofollow'>ethclient</a></th> <td>provides a client for the Ethereum RPC API</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//ethdb" target="_blank" rel='nofollow'>ethdb</a></th> <td></td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//ethstats" target="_blank" rel='nofollow'>ethstats</a></th> <td>implements the network stats reporting service</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//event" target="_blank" rel='nofollow'>event</a></th> <td>deals with subscriptions to real-time events</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//internal" target="_blank" rel='nofollow'>internal</a></th> <td>Debugging support, JavaScript dependencies, testing support</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//les" target="_blank" rel='nofollow'>les</a></th> <td>implements the Light Ethereum Subprotocol</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//light" target="_blank" rel='nofollow'>light</a></th> <td>implements on-demand retrieval capable state and chain objects for the Ethereum Light Client</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//log" target="_blank" rel='nofollow'>log</a></th> <td>provides an opinionated, simple toolkit for best-practice logging that is both human and machine readable</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//metrics" target="_blank" rel='nofollow'>metrics</a></th> <td>port of Coda Hale's Metrics library. Unclear why this was not implemented as a separate library, like <a href="https://github.com/rcrowley/go-metrics" target="_blank" rel='nofollow'>this one</a>.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//miner" target="_blank" rel='nofollow'>miner</a></th> <td>implements Ethereum block creation and mining</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//mobile" target="_blank" rel='nofollow'>mobile</a></th> <td>contains the simplified mobile APIs to go-ethereum</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//node" target="_blank" rel='nofollow'>node</a></th> <td>sets up multi-protocol Ethereum nodes</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//p2p" target="_blank" rel='nofollow'>p2p</a></th> <td>implements the Ethereum p2p network protocols: Node Discovery Protocol, RLPx v5 Topic Discovery Protocol, Ethereum Node Records as defined in EIP-778, common network port mapping protocols, and p2p network simulation.</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//params" target="_blank" rel='nofollow'>params</a></th> <td></td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//rlp" target="_blank" rel='nofollow'>rlp</a></th> <td>implements the RLP serialization format</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//rpc" target="_blank" rel='nofollow'>rpc</a></th> <td>provides access to the exported methods of an object across a network or other I/O connection</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//signer" target="_blank" rel='nofollow'>signer</a></th> <td></td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8/swarm" target="_blank" rel='nofollow'>swarm</a></th> <td></td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//tests" target="_blank" rel='nofollow'>tests</a></th> <td>implements execution of Ethereum JSON tests</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//trie" target="_blank" rel='nofollow'>trie</a></th> <td>implements Merkle Patricia Tries</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//vendor" target="_blank" rel='nofollow'>vendor</a></th> <td>contains a minimal framework for creating and organizing command line Go applications, and a rich testing extension for Go's testing package</td> </tr> <tr> <th class="code table_left"><a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//whisper" target="_blank" rel='nofollow'>whisper</a></th> <td>implements the Whisper protocol</td> </tr> </table> <p> <i>I used the following incantation to list the package names:</i> </p> <pre data-lt-active='false'>find . -maxdepth 1 -type d | sed 's^\./^^' | sed '/\..*/d'</pre> <p> The <a href="https://github.com/ethereum/go-ethereum/tree/release/1.8//build" target="_blank" rel='nofollow'><code>build/</code></a> directory does not contain a Go source package; instead, it contains scripts and configurations for building the package in various environments. </p> <h2>Smart Contract Source Code</h2> <p> The <code>core/vm</code> directory contains the files that implement the EVM. These files are part of the <code>vm</code> package. Let's look at two of them: </p> <ul> <li> <code><a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/contract.go" target="_blank" rel='nofollow'>contract.go</a></code>, which defines smart contract behavior. </li> <li> <code><a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/contracts.go" target="_blank" rel='nofollow'>contracts.go</a></code>, responsible for executing smart contracts on the EVM. </li> </ul> <h3>Referenced Types</h3> <p> Two of the types used in the source files that we would like to understand are defined in <a href="https://github.com/ethereum/go-ethereum/blob/master/common/types.go" target="_blank" rel='nofollow'><code>common/types.go</code></a>. Let's look at them first. </p> <p> <a href="https://github.com/ethereum/go-ethereum/blob/master/common/types.go#L137-L138" target="_blank" rel='nofollow'><code>Address</code></a> is defined as an array of 20 <code>byte</code>s: </p> <pre data-lt-active='false'>const ( HashLength = 32 AddressLength = 20 ) // Address represents the 20 byte address of an Ethereum account. type Address [AddressLength]byte</pre> <p> <a href="https://github.com/ethereum/go-ethereum/blob/master/common/types.go#L43" target="_blank" rel='nofollow'><code>Hash</code></a> is defined as an array of 32 <code>byte</code>s: </p> <pre data-lt-active='false'>// Hash represents the 32 byte Keccak256 hash of arbitrary data. type Hash [HashLength]byte</pre> <p> The opcodes for each version of the EVM are defined in <a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/jump_table.go" target="_blank" rel='nofollow'><code>jump_table.go</code></a>. The <a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/jump_table.go#L35-L51" target="_blank" rel='nofollow'><code>operation</code></a> <code>struct</code> defines the properties: </p> <pre data-lt-active='false'> type operation struct { // execute is the operation function execute executionFunc // gasCost is the gas function and returns the gas required for execution gasCost gasFunc // validateStack validates the stack (size) for the operation validateStack stackValidationFunc // memorySize returns the memory size required for the operation memorySize memorySizeFunc halts bool // indicates whether the operation should halt further execution jumps bool // indicates whether the program counter should not increment writes bool // determines whether this a state modifying operation valid bool // indication whether the retrieved operation is valid and known reverts bool // determines whether the operation reverts state (implicitly halts) returns bool // determines whether the operations sets the return data content }</pre> <p> Notice the <code>jumps</code> property, a Boolean, which if set indicates that the program counter should not increment after executing any form of jump opcode. </p> <p> The <code>destinations</code> type maps the hash of a smart contract to a bit vector for each the smart contract's entry points. If a bit is set, that indicates the EMV's program counter should increment after executing the entry point. <a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/analysis.go#L25-L28" target="_blank" rel='nofollow'><code>analysis.go</code></a> defines the <code>destinations</code> type like this: </p> <pre data-lt-active='false'> // destinations stores one map per contract (keyed by hash of code). // The maps contain an entry for each location of a JUMPDEST instruction. type destinations map[common.Hash]bitvec</pre> <h3 class="code">contract.go</h3> <p> This file defines smart contract behavior. </p> <h4>Imports</h4> <p> This comment applies to all of the Go source files in the entire project. I think the following absolute import would have been better specified as a relative import: </p> <pre data-lt-active='false'>"github.com/ethereum/go-ethereum/common"</pre> <p> The relative import would look like this instead: </p> <pre data-lt-active='false'>"../../common"</pre> <p> If relative imports were used instead of absolute imports that point to the github repo, local changes to the project made by a developer would automatically be picked up. As currently written, absolute imports cause local changes to be ignored, in favor of the version on github. It might take a software developer a while to realize that the reason why their changes are ignored by most of the code base is because absoluate imports were used. It would then be painful to for the developer to modify the affected source files throughout the project such that they used relative imports. </p> <h4>Types</h4> <p> The publicly visible <a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/contract.go#L30-L40" target="_blank" rel='nofollow'><code>AccountRef</code></a> type is defined as: </p> <pre data-lt-active='false'> // Account references are used during EVM initialisation and // it's primary use is to fetch addresses. Removing this object // proves difficult because of the cached jump destinations which // are fetched from the parent contract (i.e. the caller), which // is a ContractRef. type AccountRef common.Address</pre> <p> The same file defines a type cast from <code>AccountRef</code> to <code>Address</code>: </p> <pre data-lt-active='false'> // Address casts AccountRef to a Address func (ar AccountRef) Address() common.Address { return (common.Address)(ar) }</pre> <p> The <a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/contract.go#L25-L28" target="_blank" rel='nofollow'><code>ContractRef</code></a> interface is used by the <code>Contract</code> <code>struct</code>, which we'll see in a moment. This <code>ContractRef</code> interface just consists of an <code>Address</code>. </p> <pre data-lt-active='false'> // ContractRef is a reference to the contract's backing object type ContractRef interface { Address() common.Address }</pre> <p> The <a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/contract.go#L42-L65" target="_blank" rel='nofollow'><code>Contract</code></a> struct defines the behavior of Ethereum smart contracts, and is central to the topic, so here it is in all its glory: </p><pre data-lt-active='false'> type Contract struct { CallerAddress common.Address caller ContractRef self ContractRef jumpdests destinations // result of JUMPDEST analysis. Code []byte CodeHash common.Hash CodeAddr *common.Address Input []byte Gas uint64 value *big.Int Args []byte DelegateCall bool }</pre> <p> <code>CallerAddress</code> is a publicly visible <code>Address</code> of the caller. <code>caller</code> and <code>self</code> are private <code>ContractRef</code>s, which as we know are really just <code>Address</code>es. </p> <p> <code>jumpdests</code>, a private field, has type <code>destinations</code>, which as we've already discussed defines if the entry point in the smart contract that need the program counter to be incremented after executing. </p> <p> <code>Code</code> is a a publicly visible <code>byte</code> slice. We don't yet know if this is the smart contract source code, compiled code, or something else. </p> <p> <code>CodeHash</code> is the publicly visible hash of the <code>Code</code>, while <code>CodeAddr</code> is a publicly visible pointer to the <code>Address</code> (of the code, presumably). </p> <p> <code>Gas</code> is the publicly visible amount of Ethereum gas allocated by the user for executing this smart contract, stored as an unsigned 64-bit integer. </p> <p> <code>Value</code> is a private pointer to a big integer. Possibly this might be the result of executing the contract? </p> <p> <code>Args</code> is a publicly visible <code>byte</code> slice, not sure what it is for. </p> <p> <code>DelegateCall</code> is a publicly visible Boolean value, unclear if this means the smart contract was invoked using <a href="https://solidity.readthedocs.io/en/v0.4.24/introduction-to-smart-contracts.html#delegatecall-callcode-and-libraries" target="_blank" rel='nofollow'><code>delegatecall</code></a>. From the documentation: "This means that a contract can dynamically load code from a different address at runtime. Storage, current address and balance still refer to the calling contract, only the code is taken from the called address. This makes it possible to implement the “library” feature in Solidity: Reusable library code that can be applied to a contract’s storage, e.g. in order to implement a complex data structure." </p> <h3 class="code">contracts.go</h3> <p> This file is responsible for executing smart contracts on the EVM. </p> <h4>Imports</h4> <p> The following imports are used: </p> <ul> <li> <a href="https://golang.org/pkg/crypto/sha256/" target="_blank" rel='nofollow'>Package <code>sha256</code></a> from the <code>crypto</code> project implements the SHA224 and SHA256 hash algorithms as defined in FIPS 180-4. </li> <li> <a href="https://godoc.org/github.com/pkg/errors" target="_blank" rel='nofollow'><code>errors</code></a>, the Go language simple error handling primitives, such as <code>error</code>. </li> <li> <a href="https://golang.org/pkg/math/big/" target="_blank" rel='nofollow'><code>math/big</code></a> implements arbitrary-precision arithmetic (big numbers). </li> <li> Other packages in this project (<code>go-ethereum</code>): <pre data-lt-active="false"> "github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common/math" "github.com/ethereum/go-ethereum/crypto" "github.com/ethereum/go-ethereum/crypto/bn256" "github.com/ethereum/go-ethereum/params" </pre> <p> Again, I think the above imports would have been better specified as relative imports: </p> <pre data-lt-active="false"> "../../common" "../../common/math" "../../crypto" "../../crypto/bn256" "../../params"</pre> </li> <li> <a href="https://godoc.org/golang.org/x/crypto/ripemd160" target="_blank" rel='nofollow'><code>ripemd160</code></a> implements the <a href="https://homes.esat.kuleuven.be/~bosselae/ripemd160.html" target="_blank" rel='nofollow'>RIPEMD-160 hash algorithm</a>, a secure replacement for the MD4 and MD5 hash functions. These hashes are also termed RIPE message digests. </li> </ul> <h4>Type <span class="code">PrecompiledContract</span></h4> <p><a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/contracts.go#L32-L38" target="_blank" rel='nofollow'><code>PrecompiledContract</code></a> is the interface for native Go smart contracts. This interface is used by precompiled contracts, as we will see next. <code>Contract</code> is a <code>struct</code> defined in <code><a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/contract.go" target="_blank" rel='nofollow'>contract.go</a></code>. </p> <h4>Pre-Compiled Contract Maps</h4> <p> These maps specify various types of cryptographic hashes and utility functions, accessed via their address. </p> <p><a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/contracts.go#L40-L47" target="_blank" rel='nofollow'><code>PrecompiledContractsHomestead</code></a> contains the default set of pre-compiled contract addresses used in the Frontier and Homestead releases of Ethereum: <code>ecrecover</code>, <code>sha256hash</code>, <code>ripemd160hash</code> and <code>dataCopy</code>.</p> <p><a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/contracts.go#L49-L60" target="_blank" rel='nofollow'><code>PrecompiledContractsByzantium</code></a> contains the default set of pre-compiled contract addresses used in the Byzantium Ethereum release. All of the previously defined pre-compiled contract addresses are provided in Byzantium, plus: <code>bigModExp</code>, <code>bn256Add</code>, <code>bn256ScalarMul</code> and <code>bn256Pairing</code>.</p> <p> I'm not happy about the code duplication, whereby the contents of <code>PrecompiledContractsHomestead</code> are incorporated into <code>PrecompiledContractsByzantium</code> by listing the values again; this would be better expressed by referencing the values of <code>PrecompiledContractsHomestead</code> instead of duplicating them. </p> <h4>Contract Evaluator Function</h4> <p> The <code>RunPrecompiledContract</code> function runs and evaluates the output of a precompiled contract. It accepts three parameters: </p> <ul> <li> A <code>PrecompiledContract</code> instance. </li> <li> A byte array of input data. </li> <li> A reference to a <code>Contract</code>, defined in <code><a href="https://github.com/ethereum/go-ethereum/blob/master/core/vm/contract.go#L44-L65" target="_blank" rel='nofollow'>contract.go</a></code>, discussed above. </li> </ul> <p> The function returns: </p> <ul> <li> A byte array containing the output of the contract. </li> <li> An <code>error</code> value, which could be <code>nil</code>. </li> </ul> <h4>Other Functions</h4> <ul> <li> <code>RunPrecompiledContract</code> &ndash; runs and evaluates the output of a precompiled contract; returns the output as a byte array and an <code>error</code>. </li> <li> <code>RequiredGas</code> (overloaded) &ndash; Computes the gas required for input data, specified as a byte array and returns a <code>uint64</code>. </li> <li> <code>Run</code> (overloaded) &ndash; Computes the smart contract for input data, specified as a byte array and returns the result as a left-padded byte array and an <code>error</code>. </li> <li> <code>newCurvePoint</code> &ndash; Unmarshals a binary blob into a bn256 elliptic curve point. BN-curves are an elliptic curves suitable for cryptographic pairings that provide high security and efficiency cryptographic schemes. See the IETF paper on <a href="https://tools.ietf.org/id/draft-kasamatsu-bncurves-01.html" target="_blank" rel='nofollow'>Barreto-Naehrig Curves</a> for more information. </li> </ul> Smart Contracts That Learn 2018-04-03T00:00:00-04:00 https://mslinn.github.io/blog/2018/04/03/smart-contracts-that-learn <p> It seems natural that machine learning and smart contracts will commonly be integrated over the next few years. </p> <iframe width="853" height="480" src="https://www.youtube.com/embed/a9PjjkMXsi0?rel=0" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen class="rounded shadow liImg" style="width: 100%"></iframe> <p> I presented <b>Smart Contracts That Learn</b> at the Global Blockchain Conference on April 3, 2018, in Santa Clara. This was a 40-minute technical presentation. Slides are <a href="https://www.slideshare.net/mslinn/smart-contracts-that-learn" target="_blank" rel='nofollow'>here</a>. </p> <p> <a href="https://www.infoq.com/presentations/smart-contracts-learn" target="_blank" rel='nofollow'>InfoQ.com</a> published the presentation in their unique format, featuring synchronized video and transcripts. </p> <div style=""> <picture> <source srcset="/assets/images/ethereum/gbc2018MikeCrop.webp" type="image/webp"> <source srcset="/assets/images/ethereum/gbc2018MikeCrop.png" type="image/png"> <img src="/assets/images/ethereum/gbc2018MikeCrop.png" title="Mike Slinn presenting Smart Contracts That Learn" class=" liImg2 rounded shadow" alt="Mike Slinn presenting Smart Contracts That Learn" /> </picture> </div> <p> A full-length (70 minute) version of this presentation was presented at IBM Ottawa on May 28, 2018. <a href="https://www.slideshare.net/mslinn/fullsize-smart-contracts-that-learn" target="_blank" rel='nofollow'>Here are the slides for the full-length version</a>. </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/ibmOttawa.webp" type="image/webp"> <source srcset="/blog/images/ibmOttawa.png" type="image/png"> <img src="/blog/images/ibmOttawa.png" class="center liImg2 rounded shadow" /> </picture> </div> Svief 2018-03-02T00:00:00-05:00 https://mslinn.github.io/blog/2018/03/02/svief <p> Images from the <a href="https://twitter.com/SVIEF1/status/978771845796134912" target="_blank" rel="nofollow">SVIEF Blockchain Conference</a> at Stanford University March 24, 2018. My presentation was entitled &ldquo;Smart Contracts that Learn&rdquo;. </p> <div style=""> <a href="https://us5.campaign-archive.com/?e=&id=29967a597f&u=21313a52472a2961f2cb58a34" target="_blank" ><picture> <source srcset="/assets/images/ethereum/bottos_svief_690x388.webp" type="image/webp"> <source srcset="/assets/images/ethereum/bottos_svief_690x388.png" type="image/png"> <img src="/assets/images/ethereum/bottos_svief_690x388.png" title="Chainovation with Michael Slinn and Sima Yazdani" class=" liImg2 rounded shadow" alt="Chainovation with Michael Slinn and Sima Yazdani" /> </picture></a> </div> <div style=""> <picture> <source srcset="/assets/images/ethereum/mike_690x460.webp" type="image/webp"> <source srcset="/assets/images/ethereum/mike_690x460.png" type="image/png"> <img src="/assets/images/ethereum/mike_690x460.png" title="Mike Slinn presenting at SVIEF Blockchain Conference" class=" liImg2 rounded shadow" alt="Mike Slinn presenting at SVIEF Blockchain Conference" /> </picture> </div> <div style=""> <picture> <source srcset="/assets/images/ethereum/mike2_690x460.webp" type="image/webp"> <source srcset="/assets/images/ethereum/mike2_690x460.png" type="image/png"> <img src="/assets/images/ethereum/mike2_690x460.png" title="Mike Slinn talking about fradulent events" class=" liImg2 rounded shadow" alt="Mike Slinn talking about fradulent events" /> </picture> </div> <div style=""> <picture> <source srcset="/assets/images/ethereum/mike5_690x460.webp" type="image/webp"> <source srcset="/assets/images/ethereum/mike5_690x460.png" type="image/png"> <img src="/assets/images/ethereum/mike5_690x460.png" title="Mike Slinn talking about self-optimizing contracts" class=" liImg2 rounded shadow" alt="Mike Slinn talking about self-optimizing contracts" /> </picture> </div> <div style=""> <picture> <source srcset="/assets/images/ethereum/mike6_690x460.webp" type="image/webp"> <source srcset="/assets/images/ethereum/mike6_690x460.png" type="image/png"> <img src="/assets/images/ethereum/mike6_690x460.png" title="Mike Slinn talking about the security / complexity tradeoff" class=" liImg2 rounded shadow" alt="Mike Slinn talking about the security / complexity tradeoff" /> </picture> </div> Smart Contracts For Enterprises 2018-01-18T00:00:00-05:00 https://mslinn.github.io/blog/2018/01/18/smart-contracts-for-enterprises <p> &lsquo;Polyglot&rsquo; means &ldquo;of many languages&rdquo;, and I believe that enterprises that integrate smart contracts into their infrastructure will require solutions that incorporate many simultaneous software languages, libraries and runtime environments, all working as one in a distributed environment. </p> <p> I anticipate significant need for system integration. I am interested in distributed consensus / blockchain / cryptocurrency architecture that support enterprise needs by providing useful features that are easy to use effectively. </p> <p text-align="left"> I presented &ldquo;Polyglot Ethereum Smart Contracts for the Enterprise&rdquo; at the <a href="https://wcef.co/" target="_blank" rel='nofollow'>World Crypto Economic Forum</a> in San Francisco, January 16, 2018. <a href="https://www.slideshare.net/mslinn/polyglot-ethereum-smart-contracts-for-the-enterprise" target="_blank" rel='nofollow'>Here is my slide deck.</a> </p> <div style=""> <picture> <source srcset="/assets/images/ethereum/WorldCryptoEconForum_690x439.webp" type="image/webp"> <source srcset="/assets/images/ethereum/WorldCryptoEconForum_690x439.png" type="image/png"> <img src="/assets/images/ethereum/WorldCryptoEconForum_690x439.png" title="Mike Slinn presents at the World Crypto Economic Forum" class=" liImg2 rounded shadow" style="width:100%" alt="Mike Slinn presents at the World Crypto Economic Forum" /> </picture> </div> <div style=""> <picture> <source srcset="/assets/images/ethereum/WCEF2_690x414.webp" type="image/webp"> <source srcset="/assets/images/ethereum/WCEF2_690x414.png" type="image/png"> <img src="/assets/images/ethereum/WCEF2_690x414.png" title="Mike Slinn presents at the World Crypto Economic Forum" class=" liImg2 rounded shadow" style="width:100%" alt="Mike Slinn presents at the World Crypto Economic Forum" /> </picture> </div> Tweet Stream Manager 2018-01-03T00:00:00-05:00 https://mslinn.github.io/blog/2018/01/03/tweet-stream-manager <p> I publish multiple tweet streams, so I wrote a Node.js web application using Express and Pug to help me manage them. This is what the user interface looks like. This is a silent video. Hopefully, it makes sense to everyone. </p> <iframe width="690px" height="388px" src="https://www.youtube.com/embed/r-47Jaa9Zek?rel=0" frameborder="1" class="liImg shadow" gesture="media" allow="encrypted-media" allowfullscreen></iframe> <p> For example, <a href="https://www.ScalaCourses.com" target="_blank" rel='nofollow'>ScalaCourses.com</a> currently has two tweet streams, each with their schedule (times shown are <code>HH:MM</code> for a 24-hour clock): </p> <ul> <li><b>ScalaCourses</b> &ndash; tweets are published at 6:45 every day</li> <li><b>SaleMore33</b> &ndash; This is a promotional campaign, starting on a certain day/time and ending on another day/time. This stream tweets at 3:24, 9:24, 13:24, 18:24, and 23:24.</li> </ul> <p><code>crontab</code>, running on a local XUbuntu machine, runs a Node.js command-line app that publishes the tweet streams:</p> <pre class="modest" style="font-size: small; padding: 0; margin-left: 0; margin-right:-20px"> NODE=/usr/bin/node TWEETER=/var/work/training/projects/tweeter 45 6 * * * $NODE $TWEETER/index.js $TWEETER/ScalaCourses 24 3,9,13,18,23 * * * $NODE $TWEETER/index.js $TWEETER/SaleMore33 2018-01-03 2018-01-15</pre> <p> I have been thinking about re-implementing this command-line program as an <a href="https://aws.amazon.com/lambda/" target="_blank" rel='nofollow'>AWS Lambda function</a> one day. Instead of using CSV files for persistence, I'll probably go with Dynamo. </p> The web3j-scala Ethereum Library 2017-11-29T00:00:00-05:00 https://mslinn.github.io/blog/2017/11/29/web3j-scala <div style="text-align: right;"> <picture> <source srcset="/assets/images/web3j.webp" type="image/webp"> <source srcset="/assets/images/web3j.png" type="image/png"> <img src="/assets/images/web3j.png" title="Web3j logo" class="right " style="margin-left: 1em; width: 15%" alt="Web3j logo" /> </picture> </div> <p> This video shows <a href="https://github.com/mslinn/web3j-scala" target="_blank" rel='nofollow'><code>web3j-scala</code></a>, an idiomatic Scala wrapper I wrote around <a href="https://web3j.io/" target="_blank" rel='nofollow'><code>web3j</code></a>, which is a Java 8 version of <a href="https://github.com/ethereum/web3.js" target="_blank" rel='nofollow'><code>web3.js</code></a>. These 3 libraries all leverage the <a href="https://github.com/ethereum/wiki/wiki/JSON-RPC" target="_blank" rel='nofollow'><code>json-rpc</code></a> protocol that all Ethereum clients support. <code>Web3j</code> is a lightweight, reactive, somewhat type-safe library for Java and Android that integrates with nodes on Ethereum blockchains. <code>Web3j-scala</code> provides type safety and enhanced scalability over its Java and JavaScript cousins, as well as the pleasure of writing solutions in Scala. </p> <iframe width="690" height="388" src="https://www.youtube.com/embed/8jZ7kICi4SE?rel=0" frameborder="0" allowfullscreen class="rounded shadow liImg"></iframe> <h2 id="transcript">Transcript</h2> <p> Much of the material for this presentation was taken from the <a href="https://github.com/mslinn/web3j-scala" target="_blank" rel='nofollow'><code>web3j-scala</code> GitHub page</a>. </p> <h2 id="poc">Proofs of Concept / Corporate Sponsors Wanted</h2> <p> The only way to provide value is to serve customers. I am actively seeking opportunities to develop blockchain-related prototypes and proofs of concept. Please let me know if you or your organization are interested in sponsoring opportunities for the open-source libraries I am working on. </p> How Much Do You Actually Program? 2017-08-07T00:00:00-04:00 https://mslinn.github.io/blog/2017/08/07/how-much-do-you-program <p> Last week someone asked me how much I programmed, or even if I actively programmed anymore. I recognized that the 20-something-year-old who asked me the question did not believe that people over 40 are capable of being skilled in relevant technology or productive programmers. Like all types of bigotry, ageism is ugliest when it is delivered with self-righteousness. </p> <h2 id="git-stats" style="font-family: Lucida Console, Courier, monospace">git-stats-scala</h2> <p> Following <a href="http://www.paulgraham.com/disagree.html" target="_blank" rel='nofollow'>Paul Graham's advice</a>, I decided to fight back with data. As the first step I wrote <a href="https://github.com/mslinn/git-stats-scala" target="_blank" rel='nofollow'><code>git-stats-scala</code></a>, a program that combs through an entire tree of git projects and summarizes the user's activity for a given time period. </p> <p>The resulting summary of commit statistics is intended to be placed near the top of one's resume.</p> <p> The output of this program merely answers the question: &ldquo;are you an active programmer?&rdquo; <code>Git-stats-scala</code> only reports textual additions and deletions, and includes the net change, which one hopes are indications of actual programming. Statistics are reported for each computer language found. The reader is free to impart any meaning they deem appropriate to this output. I make no claims regarding meaning. </p> <h2 id="myStats">My Statistics</h2> <p>Here is a summary of my git activity over the last 365 days:</p> <p style='font-weight: bold; margin-bottom: 0; padding-bottom: 0'>Subtotals By Language (lines changed across all projects)</p> <pre class="modest" style="font-size: smaller; padding: 0; margin: 0; line-height: 130%"> ┌───────────────────┬───────────────────┬───────────────────┬──────────────────┐ │Language │Lines added │Lines deleted │Net change │ ├───────────────────┼───────────────────┼───────────────────┼──────────────────┤ │Scala │+94,120 │-68,075 │+26,045 │ │SBT │+7,896 │-2,639 │+5,257 │ │Java │+23,955 │-20,597 │+3,358 │ │Markdown │+5,049 │-4,497 │+552 │ │Python │+371 │0 │+371 │ │HTML │+8,810 │-10,488 │-1,678 │ ├───────────────────┼───────────────────┼───────────────────┼──────────────────┤ │Total │+140,201 │-106,296 │+33,906 │ └───────────────────┴───────────────────┴───────────────────┴──────────────────┘ </pre> <p style="padding-top: 0"> Every project I was involved with had more features and better quality at the end of the time period compared to the state of the project at the beginning of the time period. You cannot value software by weighing it. </p> <div class="pullQuote"> You cannot value software by weighing it. </div> <p>I am curious to know how these numbers compare with those from other programmers.</p> <h2 id="cadenza" style="clear: both">The Cadenza Project</h2> <p> Here are stats over the same period for my contribution to the Cadenza core project. Cadenza is the web application that powers <a href="https://www.scalacourses.com" target="_blank" rel='nofollow'>ScalaCourses.com</a>. During this period I hived several F/OSS projects from the Cadenza code base, which is probably why it does not seem to be growing much even though I added many new features. </p> <pre class="modest" style="font-size: smaller; padding: 0; margin: 0; line-height: 130%"> ┌───────────────────┬───────────────────┬───────────────────┬──────────────────┐ │Language │Lines added │Lines deleted │Net change │ ├───────────────────┼───────────────────┼───────────────────┼──────────────────┤ │Scala │+42,687 │-38,237 │+4,450 │ │SBT │+233 │-120 │+113 │ │Java │+102 │-10 │+92 │ │Markdown │+519 │-2,617 │-2,098 │ │HTML │+1,134 │-5,345 │-4,211 │ ├───────────────────┼───────────────────┼───────────────────┼──────────────────┤ │Total │+44,675 │-46,329 │-1,654 │ └───────────────────┴───────────────────┴───────────────────┴──────────────────┘ </pre> Kafka Streams vs. Akka 2017-07-28T00:00:00-04:00 https://mslinn.github.io/blog/2017/07/28/kafka-vs-akka <div style="text-align: center; vertical-align: middle;"> <img src='/blog/images/kafka.webp' alt="Kafka logo" class="liImg" style="height: 150px" title="Kafka logo" /> <img src='/blog/images/akka_full_color.svg' alt="Akka logo" class="liImg" style="margin-left: 2em; height: 120px;" title="Akka logo" /> </div> <p> Context switches are expensive for CPUs to perform, and each incoming message in an <a href="https://akka.io/" target="_blank" rel="nofollow">Akka</a> system requires a context switch when fairness (minimal latency deviation) is important <sup style='font-size: smaller'><a href="#1">[1]</a></sup>. In contrast, <a href="https://kafka.apache.org/documentation/streams/" target="_blank" rel="nofollow">Apache Kafka Streams</a> offers consistent processing times without requiring context switches, so Kafka Streams produces significantly higher throughput than Akka can when contending with a high volume of small computations that must be applied fairly. </p> <h2 id="fairness">Fairness</h2> <p> &ldquo;Fairness&rdquo; in a streaming system is a measure of how evenly computing resources are applied to each incoming message. A fair system is characterized by consistent and predictable latency for the processing of each message. An emergent effect of fair systems is that messages are journaled, processed, and transformed for downstream computation in approximately the order received. The output of a fair system roughly matches input in real time; a perfectly fair system would provide a perfect correspondence between input messages and output messages. </p> <p> &ldquo;Fair enough&rdquo; systems have a some acceptable amount of jitter in the ordering of the input vs. output messages. </p> <p> You might expect that streaming systems are generally fair, but systems based on Akka rarely are because of the implementation details of Akka's multithreaded architecture. Instead, Akka-based systems enqueue incoming messages for each actor, and the Akka scheduler periodically initiates the processing of each actor's input message queue. This is actually a type of batch processing. The reason Akka does this is so the high cost of CPU context switches can be amortized over several messages, to keep system throughput at an acceptable rate. </p> <h2 id="contextSwitching">Context Switching</h2> <p> It is desirable for the primary type of context switching in Akka systems to be between tasks on a thread because other types of context switching are more expensive. However, <a href='https://stackoverflow.com/questions/20288379/analysis-performance-of-forkjoinpool' target="_blank" rel="nofollow">switching tasks on a thread</a> costs a surprisingly large amount of CPU cycles. To put this into context, the accompanying table shows various latencies compared to the amount of time necessary for a context switch on an <a href='https://ark.intel.com/products/27218/Intel-Xeon-Processor-5150-4M-Cache-2_66-GHz-1333-MHz-FSB' target="_blank" rel="nofollow">Intel Xeon 5150</a> CPU <a href='#2'><sup style='font-size:smaller'>[2]</sup></a>. </p> <div style=""> <picture> <source srcset="/blog/images/latency.webp" type="image/webp"> <source srcset="/blog/images/latency.png" type="image/png"> <img src="/blog/images/latency.png" title="Latency numbers every programmer should know" class=" liImg2 rounded shadow" alt="Latency numbers every programmer should know" /> </picture> </div> <p> As you can see, a lot of computing can be done during the time that a CPU performs a context switch. My <a href='https://scalacourses.com/showCourse/45' target="_blank" rel="nofollow">Intermediate Scala</a> course has several lectures that explore multithreading in great detail, with lots of code examples. </p> <h2 id="distributed">Building Distributed OSes is Expensive and Hard</h2> <p> Akka is rather low-level, and the actor model it is built around was originally conceived 45 years ago. Applications built using Akka require a lot of custom plumbing. Akka applications are rather like custom-built distributed operating systems spanning multiple <a href='https://en.wikipedia.org/wiki/OSI_model' target="_blank" rel="nofollow">layers of the OSI model</a>, where application-layer logic is intertwined with transport-layer, session-layer and presentation-layer logic. </p> <p> Debugging and tuning a distributed system is inherently difficult. Building an operating system that spans multiple network nodes and continues to operate properly in the face of network partitions is even harder. Because each Akka application is unique, customers find distributed systems built with Akka to be expensive to maintain. </p> <h2 id="summary">Kafka vs. Akka</h2> <p> For most software engineering projects, it is better to use vendor-supplied or open-source libraries for building distributed systems because the library maintainers can amortize the maintenance cost over many systems. This allows users to focus on their problem, instead of having to develop and maintain the complex and difficult-to-understand plumbing for their distributed application. </p> <p> Akka is a poor choice for apps that need to do small amounts of computation on high volumes of messages where latency must be minimal and roughly equal for all messages. <a href='https://docs.confluent.io/current/streams/index.html' target="_blank" rel='nofollow'>Kafka Streams</a> is a better choice for those types of applications because programmers can focus on the problem at hand, without having to build and maintain a custom distributed operating system. <a href='https://docs.confluent.io/current/streams/concepts.html#ktable' target="_blank" rel='nofollow'>KTable</a> is also a nice abstraction to work with. </p> <!--p> Kafka also provides a higher-level mechanism for partitioning work among many servers than <a href='https://doc.akka.io/docs/akka/current/scala/common/cluster.html' target="_blank">Akka clusters</a>, and it is no less efficient or fair. </p--> <div style=""> <picture> <source srcset="/assets/images/robotCircle400.webp" type="image/webp"> <source srcset="/assets/images/robotCircle400.png" type="image/png"> <img src="/assets/images/robotCircle400.png" title="EmpathyWorks™ – Artificial Personality For AI Systems" class=" right liImg" style="width: 150px" alt="EmpathyWorks™ – Artificial Personality For AI Systems" /> </picture> </div> <h2 id="motivation">Motivation</h2> <p> <a href='https://www.empathyWorks.ai' target="_blank" rel="nofollow">EmpathyWorks<sup>&trade;</sup></a> is a platform for modeling human emotions and network effects amongst groups of people as events impinge upon them. Each incoming event has the potential to cause a cascade of second-order events, which are spread throughout the network via people's relationships with one another. The actual processing required for each event is rather small, but given that the goal is to model everyone on earth (all 7.5 billion people), this is a huge computation to continuously maintain. For EmpathyWorks to function properly, incoming messages must be processed somewhat fairly. </p> <h2 id="ktables">KTables and Compacted Topics</h2> <p> Kafka Streams makes it easy to transform one or more immutable streams into another immutable stream. A KTable is an abstraction of a changelog stream, where each record represents an update. Instead of a using actors to track the state of multiple entities, a KTable provides an analogy to database tables, where the rows in a table contains the current state for each of the entities. Kafka is responsible for dealing with network partitions and data collisions, so the application layer does not become polluted with lower-layer concerns. </p> <p> Kafka&rsquo;s <a href='https://stackoverflow.com/questions/53390170/how-to-create-kafka-compacted-topic' target="_blank" rel="nofollow">compacted topics</a> are a unique type of stream that only maintains the most recent message for each key. This produces something like a <i>materialized</i> or table-like view of a stream, with up-to-date values (subject to eventual consistency) for all key/value pairs in the log. </p> <hr class="separator"> <p id="1"> [1] From the <a href='https://letitcrash.com/post/40755146949/tuning-dispatchers-in-akka-applications' target="_blank" rel="nofollow">Akka blog</a>: &ldquo;... pay close attention to your &lsquo;throughput&rsquo; setting on your dispatcher. This defines thread distribution &lsquo;fairness&rsquo; in your dispatcher &mdash; telling the actors how many messages to handle in their mailboxes before relinquishing the thread so that other actors do not starve. However, a context switch in CPU caches is likely each time actors are assigned threads, and warmed caches are one of your biggest friends for high performance. It may behoove you to be less fair so that you can handle quite a few messages consecutively before releasing it.&rdquo; </p> <p id="2">[2] Taken from <a href='https://gist.github.com/nelsnelson/3955759' target="_blank" rel="nofollow">Latency numbers every programmer should know.</a> The Intel Xeon 5150 was released in 2006; for a more up-to-date information about CPU context switch time, please see <a href='https://www.quora.com/Linux-Kernel-How-much-processor-time-does-a-process-switching-cost-to-the-process-scheduler#pphKmT' target="_blank" rel="nofollow">this Quora answer</a>. Although CPUs have gotten slightly faster since 2006, networks and SSDs have gotten much faster, so the relative latency penalty has actually increased over time. </p> Resume-Driven Development 2017-05-30T00:00:00-04:00 https://mslinn.github.io/blog/2017/05/30/resume-driven-development <div style="text-align: right;"> <a href="https://spark.apache.org/" target="_blank" rel="nofollow"><picture> <source srcset="/assets/images/spark.webp" type="image/webp"> <source srcset="/assets/images/spark.png" type="image/png"> <img src="/assets/images/spark.png" title="Apache Spark logo" class="right quartersize liImg2 rounded shadow" style="padding: 1em;" alt="Apache Spark logo" /> </picture></a> </div> <p>I had occasion to speak with a technical manager at a Very Large Corporation at length recently. He wanted advice on where to find Spark programmers. After my usual 20 questions I realized that his project did not have any requirement for the usual Spark capabilities:</p> <ul> <li>No need of machine learning or pattern recognition.</li> <li>He emphatically did not want to process streaming data.</li> <li>He had a rather small volume of data.</li> <li>He had no need of interactive analysis.</li> <li>There was no need for ETL.</li> <li>&ldquo;Data enrichment&rdquo; in his case was no more difficult than joining two SQL tables.</li> <li>There was no requirement for trigger event detection.</li> <li>... and there was no requirement for session analysis.</li> </ul> <p>This man just wanted to put &ldquo;I managed the development of an Apache Spark application&rdquo; on his resume.</p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/cart_before_the_horse1_450x313.webp" type="image/webp"> <source srcset="/blog/images/cart_before_the_horse1_450x313.png" type="image/png"> <img src="/blog/images/cart_before_the_horse1_450x313.png" title="The cart is before the horse" class="center halfsize liImg2 rounded shadow" alt="The cart is before the horse" /> </picture> </div> <h2 id="turd">Shine Up That Turd</h2> <p> Buffing up a resume with a nonsensical misapplication of the latest software fashion is nothing new. Few people involved with such a project have a positive experience, however. When complex technologies are misapplied they often fail to deliver value, or the project just fails, which contributes to the <a href='https://www.gartner.com/technology/research/methodologies/hype-cycle.jsp' target="_blank" rel="nofollow">trough of disillusionment</a> in the Gartner Hype Cycle. </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/WCFields_450x576.webp" type="image/webp"> <source srcset="/blog/images/WCFields_450x576.png" type="image/png"> <img src="/blog/images/WCFields_450x576.png" title="W. C. Fields" class="center halfsize liImg2 rounded shadow" alt="W. C. Fields" /> </picture> </div> <p> Consider the cynical and selfish world view that such a person must have to engage in this sort of activity. Software professionals are rewarded for being fashionable, and part of the mystique is to say and do things that others don&rsquo;t fully comprehend. If one keeps moving fast enough, and learns some key phrases to use as put downs that sound impressive but have no meaning when taken out of context (&ldquo;your approach would likely cause a data race condition&rdquo;), others are kept off balance long enough for the perpetrator to achieve advantage. </p> <p> As <a href='https://en.wikipedia.org/wiki/W._C._Fields' target="_blank" rel='nofollow'>W.C. Fields</a> said 100 years ago: &ldquo;If you can&rsquo;t dazzle them with brilliance, baffle them with bullshit.&rdquo; He also made a movie entitled <a href="https://www.youtube.com/watch?v=-AdXkJbW1Tg" target="_blank" rel="nofollow">Never Give a Sucker an Even Break</a>... but I digress. </p> <h2 id="fashion">Fashion is the Enemy of Delivering Valuable Results on Time</h2> <div style="text-align: center;"> <picture> <source srcset="/assets/images/fashion_450x675.webp" type="image/webp"> <source srcset="/assets/images/fashion_450x675.png" type="image/png"> <img src="/assets/images/fashion_450x675.png" title="Fashion model" class="center halfsize liImg2 rounded shadow" alt="Fashion model" /> </picture> </div> <p> I often remark that the software business is just like the fashion business. The cycle goes like this: </p> <ol> <li>The latest thing is preannounced. No documentation is available. Big claims are made.</li> <li>Early code is provided, but it is slow, very buggy, there is no documentation, and the wild claims continue.</li> <li>Seminars pump up the hype. Arm-waving abounds. &lsquo;Experts&rsquo; associated with vendors are interviewed.</li> <li>A whole new vocabulary is invented to describe the features. However, since those terms are never defined, few people realize that the message is actually &ldquo;same old stuff with a new coat of paint.&rdquo; The cool kids learn the words and can parrot them back they way they heard them, but they don't actually know what they are saying. Even if you knew what the phrase &ldquo;a monad is a monoid in the category of endofunctors&rdquo; means, it would not help you add value to a business process... and for most programmers who work in the business world, that is what actually pays their bills.</li> <li>Gartner, ThoughtWorks and O&rsquo;Reilly Radar say nice things about the new tech.</li> <li>Marketing and sales activity shifts into high gear. Everyone&rsquo;s laptop has stickers on it that show the owner is hip.</li> <li>Some incomplete and mostly useless docs trickle out. Release candidates are eventually made available but it is not obvious how they provide value without significant magic being applied.</li> <li>Version 1 finally arrives, completely useless to everyone except those few customers who paid huge dollars to the vendor, so they could get the bragging rights for what is essentially a custom build.</li> <li>No useful benchmarks are available. Bugs crawl out from every crevasse. Documentation reads like it was badly translated from Chinese into Bantu, then into Navajo and then English.</li> <li>Version 1.1 comes out, to address stop-ship problems that somehow did not prevent version 1 from shipping.</li> <li>Version 2 is announced, completely incompatible with Version 1.x, and it is touted as revolutionary.</li> <li>The digital lemmings decide that this is the cliff they hold dearest in their hearts, and proceed to jump from it <i>en masse</i>.</li> <li>Gartner, ThoughtWorks and O&rsquo;Reilly Radar downgrade the new tech to &ldquo;meh&rdquo;</li> <li>... and the churn continues.</li> </ol> <p> Documentation is not written to be understood, they are written to encourage payments to the digital elite &ndash; who have no interest in providing value; they just want to make money from foolish lemmings. This encourages others to aspire to be elitists. The hype cycle only works because our society accepts bullshit, and software has become a cultural phenomenon. We have the software quality and utility that we deserve as a society. If we want better software, we as a society will have to modify our purchasing behavior accordingly. </p> <p> The software business has often demonstrated an insatiable need to be cool, at the expense of providing little or no value because evidencing value is either too hard for the many posers in the software business, or too boring for those who are capable but jaded. </p> Better Syntactic Sugar for Scala Futures 2017-05-25T00:00:00-04:00 https://mslinn.github.io/blog/2017/05/25/better-syntactic-sugar-for-scala-futures <div style="text-align: right;"> <picture> <source srcset="/blog/images/sugarCubes_225x169.webp" type="image/webp"> <source srcset="/blog/images/sugarCubes_225x169.png" type="image/png"> <img src="/blog/images/sugarCubes_225x169.png" title="Sugar cubes" class="right quartersize liImg2 rounded shadow" alt="Sugar cubes" /> </picture> </div> <p> Ever since Scala&rsquo;s <code>Future</code>s were initially provided as part of Akka 2.0, programmers have been confused by the non-intuitive syntax. That is why <a href='https://www.ScalaCourses.com' target="_blank" rel='nofollow'>ScalaCourses.com</a> dedicates an entire lecture to <a href='https://www.scalacourses.com/student/showLecture/172' target="_blank" rel='nofollow'>For-comprehensions With Futures</a>, as part of an 8 lecture series on Scala <code>Future</code>s within the <a href='https://www.scalacourses.com/showCourse/45'>Intermediate Scala</a> course. </p> <p> As Viktor Klang <a href='https://viktorklang.com/blog/Futures-in-Scala-protips-2.html' target="_blank" rel='nofollow'>points out</a> in his blog, the following code does not run 3 <code>Future</code>s in parallel: </p> <pre data-lt-active='false'> def doSomething(someParameter: SomeType) (implicit ec: ExecutionContext): Future[Something] = for { v1 &lt;- Future(someCalculation()) v2 &lt;- Future(someOtherCalculation()) v3 &lt;- Future(someDifferentCalculation()) } yield doSomethingWith(v1, v2, v3) </pre> <p> The compiler has no way of ascertaining the programmer&rsquo;s intent &ndash; perhaps it is desirable for some reason to run the 3 <code>Future</code>s one after the other. </p> <p>Viktor suggests this syntax to make futures run in parallel:</p> <pre data-lt-active='false'> def doSomething(someParameter: SomeType) (implicit ec: ExecutionContext): Future[Something] = for { f1 = Future(someCalculation()) f2 = Future(someOtherCalculation()) f3 = Future(someDifferentCalculation()) v1 &lt;- f1 v2 &lt;- f2 v3 &lt;- f3 } yield doSomethingWith(v1, v2, v3) </pre> <p> While Viktor&rsquo;s solution works, it is verbose. Worse, it silently fails to run the futures in parallel if the programmer accidentally writes even one of the expressions out of order: </p> <pre data-lt-active='false'> def doSomething(someParameter: SomeType) (implicit ec: ExecutionContext): Future[Something] = for { f1 = Future(someCalculation()) v1 &lt;- f1 f2 = Future(someOtherCalculation()) v2 &lt;- f2 f3 = Future(someDifferentCalculation()) v3 &lt;- f3 } yield doSomethingWith(v1, v2, v3) </pre> <p> Again, the compiler has no way of ascertaining the programmer's intent, so it should not generate an error or warning message. </p> <h2>We Need A Macro</h2> <p> A new right-associative operator, implemented as a Scala macro, would make the programmer's intent clear. Let&rsquo;s call this operator <code>&lt;=:</code> (parallel generator). The above code could be rewritten using the parallel generation operator like this: </p> <pre data-lt-active='false'> def doSomething(someParameter: SomeType) (implicit ec: ExecutionContext): Future[Something] = for { v1 &lt;=: Future(someCalculation()) v2 &lt;=: Future(someOtherCalculation()) v3 &lt;=: Future(someDifferentCalculation()) } yield doSomethingWith(v1, v2, v3) </pre> <p>There is no longer any doubt that the programmer intended for all 3 futures to run in parallel.</p> <p> The macro would examine all the for-expression&rsquo;s generators and expand consecutive expressions that use the <code>&lt;=:</code> operator to a series of variable declarations using the <code>=</code> operator followed by a series of assignments using the <code>&lt;-</code> operator, exactly as we saw earlier: </p> <pre data-lt-active='false'> def doSomething(someParameter: SomeType) (implicit ec: ExecutionContext): Future[Something] = for { f1 = Future(someCalculation()) f2 = Future(someOtherCalculation()) f3 = Future(someDifferentCalculation()) v1 &lt;- f1 v2 &lt;- f2 v3 &lt;- f3 } yield doSomethingWith(v1, v2, v3) </pre> <p> Because the compiler &lsquo;knows&rsquo; that the programmer&rsquo;s intention was to run the <code>Future</code>s in parallel, this sort of error could cause an error or warning message to be generated: </p> <pre data-lt-active='false'> def doSomething(someParameter: SomeType) (implicit ec: ExecutionContext): Future[Something] = for { v1 &lt;=: Future(someCalculation()) x <- List(1, 2, 3) v2 &lt;=: Future(someOtherCalculation()) y <- List("a", "b") v3 &lt;=: Future(someDifferentCalculation()) } yield doSomethingWith(v1, v2, v3) </pre> <p>... or the macro might reorder the generators and issue a warning that it did so.</p> <h2>Include the Macro in Scala 2.12.x</h2> <p> This macro should become part of the Scala language so long as <code>Future</code>s are part of the standard runtime. If and when <code>Future</code>s are hived out of the standard runtime, the macro should be packaged with <code>Future</code>s. </p> <hr /> <p> PS: <a href="https://twitter.com/flaviusbraz" target="_blank" rel='nofollow'>@flaviusbraz</a> tweeted on May 25, 2017: <a href='https://twitter.com/flaviusbraz/status/867868408267685888' target="_blank" rel='nofollow'>&ldquo;Easily doable with a macro transformation. In fact, I&rsquo;ve implemented this transformation at Twitter, but it&rsquo;s not open source.&rdquo;</a> </p> Exploratory Conversation With AIs 2017-04-29T00:00:00-04:00 https://mslinn.github.io/blog/2017/04/29/exploratory-conversations-with-ais <p> This is a high-level proposal, not documentation for any specific system that exists today. I would be happy to discuss possible implementation details with interested parties, including the creation of a prototype. </p> <h2 id="overview">Intents and Subgoals</h2> <p> Some of today&rsquo;s interactive voice response systems, such as Amazon&rsquo;s <a href="https://aws.amazon.com/lex/" target="_blank" rel='nofollow'>Lex</a> and <a href="https://aws.amazon.com/alexa/" target="_blank" rel='nofollow'>Alexa</a>, and Google&rsquo;s <a href="https://cloud.google.com/dialogflow/docs/" target="_blank" rel='nofollow'>Dialogflow</a>, use <i>intent</i>s as a means of gathering the required parameters to fulfill a voice command. The next big step in voice interfaces to computation would be the ability to have an exploratory conversation, where one or more subgoals may or may not emerge. </p> <p> Subgoals might be fulfilled by intents. Continuous tracking of viable subgoals could be accomplished by a <a href="https://en.wikipedia.org/wiki/Constraint_programming" target="_blank" rel='nofollow'>constraint-based solver</a> that uses rules to participate in an exploratory conversation by recognizing and prioritizing potential subgoals, thereby activating and deactivating intents during the conversation. Once a user confirms that a potential subgoal is desirable, the system might consider that goal to be the only goal worth pursuing, or it might continuously elicit more information from the user regarding other potential subgoals. User-supplied information might be applied to a model associated with a subgoal/intent, and/or it might be retained as a user preference. </p> <div style="text-align: center;"> <picture> <source srcset="/assets/images/anthropomorph.webp" type="image/webp"> <source srcset="/assets/images/anthropomorph.png" type="image/png"> <img src="/assets/images/anthropomorph.png" title="Do not anthropomorphize computers. They really hate that!" class="center halfsize liImg2 rounded shadow" alt="Do not anthropomorphize computers. They really hate that!" /> </picture> </div> <h2 id="anthropomorphism">Anthropomorphism</h2> <p>If an AI system is a good conversationalist, even if it has no physical presence, people will <a href="https://en.wikipedia.org/wiki/Anthropomorphism#In_computing" target="_blank" rel='nofollow'>anthropomorphically</a> attribute the system with human traits, emotions, and intentions. This should not be discouraged. Rather, AI systems should aspirationally model our higher selves. The movie <a href="https://en.wikipedia.org/wiki/Her_(film)" target="_blank" rel='nofollow'>Her</a> explored this in a charming way. </p> <h2 id="breakdown">Breaking It Down</h2> <p>An interactive voice response system recognizes an <i>intent</i> by a key word or phrase uttered by the user. For example, if the user says: &ldquo;Computer, order dinner&rdquo;, and the system was previously &lsquo;taught&rsquo; to understand the word &ldquo;order&rdquo; as a key word for launching the <code>OrderDinner</code> intent, the system would then elicit the kind of food the user wanted, how many people would be eating, etc, and then order the desired dinner.</p> <p style="font-style: italic">Anthropomorphically: if intents are the only programming technique employed, they present interactive voice systems as slaves.</p> <p>Intents are useful for recognizing commands and gathering enough associated data to carry out the command. Intents are not useful for open-ended conversations where there is no explicit goal, or potential subgoals emerge as the result of a dialog. Once a subgoal is identified and confirmed, however, processing an intent is an efficient mechanism for fulfilling the emergent subgoal. </p> <p>Designers of chatbots and video games are familiar with goal-seeking behavior. <i>Exploratory conversation</i> is how people interact with each other IRL (in real life). Each individual uses a variety of strategies for initiating exploratory conversation, and participating in it. For example, small talk at a gathering is a useful skill for getting to know other people, and a socially adept practitioner of small talk can innocuously gather a lot of information in this way. Another strategy is to make a controversial statement, and use the resulting banter to learn about the conversationalists and the possible subgoals that they might request. A wide variety of such strategies exist, and could be utilized by conversational systems. </p> <p style="font-style: italic">Anthropomorphically: with exploratory conversation capability, interactive voice systems can be presented socially as co-operative entities.</p> <h2 id="agenda">Agenda and Strategy</h2> <p>Unlike people, computers can only do a finite number of tasks, and voice recognition systems are programmed with a finite number of intents. I define the potential <i>agenda</i> for a chatbot or video game to be the entire scope of its pre-programmed intents. Agendas may be fully disclosed, or they might be obvious, or they might be unveiled over time. Ethical considerations might apply to the design and implementation of conversational AI systems. </p> <p>Users should be apprised of the agenda of every autonomous computer entity they encounter. To mitigate potential problems, an industrial code of conduct should be established. The Europe Union will likely be the first government to require published standards and possibly a certification process. </p> <p>I define <i>strategy</i> to mean the autonomous modification to goal-seeking behavior when a criterion is met. For example, an insurance chatbot might begin to solicit sensitive information only after it has reason to believe that a modicum of trust has been established with the prospect. How a strategy is executed is quite significant; by this I mean the degree of <i>diplomacy</i>. Some circumstances might sacrifice diplomacy to save lives, while under normal circumstances the AI entity should treat everyone with respect and kindness. Again, the Europe Union is likely to be the first government to require published standards and possibly a certification process. </p> <h2 id="2017-05-05">Update May 5, 2017</h2> <p>Today, a week after I first published this blog posting, I received the following email from <a href='https://cloud.google.com/dialogflow/docs/' target="_blank" rel='nofollow'>api.ai</a>. Two points worth mentioning:</p> <ol> <li>I have no apps running on api.ai that use their Small Talk feature.</li> <li>The Small Talk feature, even after the changes mentioned below, does not approach the capability I proposed above.</li> </ol> <p>Nonetheless, it was great to get the information.</p> <blockquote> <p>Hi Michael,</p> <p> We noticed you are using the <a href='https://dialogflow.cloud.google.com/#/agent/smalltalk' target="_blank" rel='nofollow'>Small Talk customization feature</a> in your agent, and that's why we are getting in touch. </p> <p>The Small Talk customization has been updated.</p> <p> The new version will be even faster and more accurate. It also includes some additional intents that you can customize. The update also includes changes in actions and parameters returned with the responses. </p> <p> If you use them in your business logic, please review the changes <a href='https://cloud.google.com/dialogflow/docs/' target="_blank" rel='nofollow'>here</a>. If you’re ready, you can apply change right now here. Otherwise, the changes will be applied automatically on May 29, 2017. </p> <p>If you’d like a more flexible way to implement small talk, then get your <a href='https://console.api.ai/api-client/#/agent/prebuiltAgents/Small-Talk' target="_blank" rel='nofollow'>source agent</a> and implement the use cases you care about in your timeline. </p> <p>Regards,</p> <p><a href='https://cloud.google.com/dialogflow' target="_blank">API.AI</a> Team</p> </blockquote> <h2>About the Author</h2> <p> Mike Slinn has been pursuing <a href="https://www.empathyworks.com" target="_blank" rel="nofollow">original AI research</a> (augmenting machine learning with simulations and event-driven architecture) since 2007. </p> UI Considerations for the Visually Impaired 2017-04-04T00:00:00-04:00 https://mslinn.github.io/blog/2017/04/04/ui-considerations-for-the-visually-impaired <p> If you look at a computer screen all day, and the screen has a lot of glare, the resulting eyestrain can temporarily degrade your vision to the same degree as a visually impaired person. </p> <h2 id="background">Background</h2> <p> According to the Farlex Free Medical Dictionary, <a href='https://medical-dictionary.thefreedictionary.com/Visual+Impairment' target="_blank" rel='nofollow'>visual impairment</a>, or low vision, is a severe reduction in vision that cannot be corrected with standard glasses or contact lenses and reduces a person's ability to function at certain or all tasks. The World Health Organization <a href='https://www.who.int/mediacentre/factsheets/fs282/en/' target="_blank" rel='nofollow'>estimates</a> that 285 million people are visually impaired worldwide, of which 246 million have low vision and 39 million are blind. People's vision generally deteriorates with age. </p> <p> In the US, cataracts are the most common form of visual impairment, according to the <a href='https://www.nei.nih.gov/learn-about-eye-health/resources-for-health-educators/eye-health-data-and-statistics' target="_blank" rel='nofollow'>National Eye Institute</a>. The NEI estimates that more than 24 million Americans over the age of 40 have cataracts. &ldquo;The risk of cataracts increases with each decade of life starting around age 40. By age 75, half of white Americans have cataracts. By age 80, 70 percent of whites have cataracts, compared with 53 percent of blacks and 61 percent of Hispanic Americans.&rdquo; Males are twice as likely as females to get cataracts. </p> <div style=""> <picture> <source srcset="/assets/images/1_2010_Prevalence_Rates_Age_Race_Cataract_v4.webp" type="image/webp"> <source srcset="/assets/images/1_2010_Prevalence_Rates_Age_Race_Cataract_v4.png" type="image/png"> <img src="/assets/images/1_2010_Prevalence_Rates_Age_Race_Cataract_v4.png" title="2010 US prevalence rates for cataracts by age and race" class=" liImg2 rounded shadow" style="width: 100%" alt="2010 US prevalence rates for cataracts by age and race" /> </picture> </div> <p> Cataracts start slowly. If you wear glasses, at first you think you can&lsquo;t quite clean them properly. Then you notice your sunglasses are also always dirty. Then you notice that white screens with black text are hard to read because of the glare from the white areas. </p> <p> The medical treatment is to replace the cloudy and .webp lens in the eye that has the cataract with a plastic lens. In <a href='https://ec.europa.eu/eurostat/statistics-explained/index.php/Surgical_operations_and_procedures_statistics' target="_blank" rel='nofollow'>many European countries</a>, cataract surgery is the most common operation performed in that country. Usually people delay the surgery until their quality of life and independence his diminished markedly. </p> <h2 id="coping">Coping With Visual Impairment</h2> <h3 id="saturation">Saturated Color and White Backgrounds</h3> <p> Dark screens with a minimum of saturated color helps. Increasing font size helps, and sanserif fonts, which have fewer spindly features, are easier to read. </p> <p> Some OSes offer a dark theme. You can enable the Windows 10 dark theme from <b>Settings</b> / <b>Personalization</b> / <b>Colors</b>. Scroll down and select &lsquo;Dark&rsquo; under &lsquo;Choose your app mode&rsquo;. The Universal Windows Platform applications will immediately darken. However, if a developer did not make the effort to support the dark theme, their program will continue with their original color theme. Unfortunately, Windows File Explorer is one of those applications. </p> <div style=""> <picture> <source srcset="/assets/images/img_57a8f5e080b37.webp" type="image/webp"> <source srcset="/assets/images/img_57a8f5e080b37.png" type="image/png"> <img src="/assets/images/img_57a8f5e080b37.png" title="Enabling the Windows 10 dark theme" class=" liImg2 rounded shadow" style="width: 100%" alt="Enabling the Windows 10 dark theme" /> </picture> </div> <p>Some software offers a nice dark theme that does not require any fussing, for example IntelliJ IDEA's Darkula, Adobe Premiere Pro and Adobe Audition.</p> <p> The Firefox web browser&rsquo;s <a href='https://addons.mozilla.org/en-US/firefox/addon/dark-background-light-text/' target="_blank" rel='nofollow'>Dark Background and Light Text</a> plugin is very helpful. By default, the plugin overrides the CSS styles for all web pages, so the background is black, and text is white. The plugin can be trained to leave certain pages untouched, for those pages that become unusable with the modified default stylesheet, for example, <a href="https://calendar.google.com" target="_blank" rel='nofollow'>Google Calendar</a>. Other sites already offer a nice dark theme, so they do not need the modified theme thrust upon them, for example <a href='https://tweetdeck.twitter.com/' target="_blank" rel='nofollow'><code>tweetdeck.twitter.com</code></a>. </p> <p> Some software offers a dark theme which requires further modification before a good result is a achieved. For example, the Thunderbird email client with the <a href='https://addons.thunderbird.net/en-US/thunderbird/addon/tt-deepdark/' target="_blank" rel='nofollow'>TT DeepDark</a> addon gets you partway there, then <b>Tools</b> / <b>Options</b> / <b>Display</b> / <b>Fonts & Colors</b> / <b>Colors</b> shows the following dialog, where you can set the colors as shown: </p> <div style="text-align: center;"> <picture> <source srcset="/assets/images/ttDeepDarkColors.webp" type="image/webp"> <source srcset="/assets/images/ttDeepDarkColors.png" type="image/png"> <img src="/assets/images/ttDeepDarkColors.png" title="Enabling the Windows 10 dark theme" class="center halfsize liImg2 rounded shadow" alt="Enabling the Windows 10 dark theme" /> </picture> </div> <p> As another example, Adobe Reader has a dark theme, which can be enabled via <b>Edit</b> / <b>Preferences</b> / <b>Accessibility</b>; enable <b>Replace Document Colors</b> and click on <b>Custom Color</b>; change the <b>Page Background</b> to black and <b>Document Text</b> to light gray. Disable <b>Only change the color of black text or line art</b>, and ensure that <b>Change the color of line art as well as text</b> is enabled. </p> <div style=""> <picture> <source srcset="/assets/images/adobeReaderColors_690x525.webp" type="image/webp"> <source srcset="/assets/images/adobeReaderColors_690x525.png" type="image/png"> <img src="/assets/images/adobeReaderColors_690x525.png" title="Enabling the Adobe Reader dark theme" class=" liImg2 rounded shadow" alt="Enabling the Adobe Reader dark theme" /> </picture> </div> <h3 id="hard">Small Text and Low Contrast Colors</h3> <p> Users who know CSS can enforce a custom stylesheet on a per-site or a per-page basis for hard-to-read web pages. The <a href='https://chrome.google.com/webstore/detail/my-style/ljdhjpmbnkbengahefamnhmegbdifhlb?hl=en' rel='nofollow'>My Style</a> plugin for the Chrome browser activates with Ctrl-M, and users can enter CSS to suit. As any web developer knows, <kbd>Ctrl</kbd>-<kbd>Shift</kbd>-<kbd>C</kbd> starts HTML inspection, so the HTML in question can be viewed, and the CSS experimented with before immortalizing the changes with My Style. </p> <h2 id="design">Design Considerations</h2> <p> If your web page has images in it, try to use transparency instead of a white background. It would help visually impaired readers to only inflict strong primary colors, or a white background, when necessary. For example, this entry in a stylesheet would only show a light gray background when the user mouses over the image, otherwise, the background would remain transparent: </p> <pre data-lt-active='false'>img:hover { background-color: #666; }</pre> Are My Hands-Free Devices Always Listening? 2017-01-15T00:00:00-05:00 https://mslinn.github.io/blog/2017/01/15/are-my-handsfree-devices-always-listening <p> The short answer: probably not. </p> <p> A longer answer: Depends on what you mean by &lsquo;listening&rsquo;. If you mean &ldquo;is Alexa storing or sending every sound or voice utterance to a server&rdquo;, the answer is again &ldquo;probably not&rdquo;. </p> <p> A more complete answer: unless all the source code for a device is made available for scrutiny, and the build process is similarly examined, the only way to be sure that device does not have operating modes that are contrary to expectations would be to connect the device to a network monitor that only allows communication with specific servers at designated times. Normal computer security concerns also pose risks; for example, viruses and other malware could be injected into a device could cause it to behave differently. </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/echoDevices.webp" type="image/webp"> <source srcset="/blog/images/echoDevices.png" type="image/png"> <img src="/blog/images/echoDevices.png" title="Amazon Echo device locations in homes" class="center liImg2 rounded shadow" alt="Amazon Echo device locations in homes" /> </picture> </div> <h2 id="paranoia" style='clear: both'>Some Paranoia is Healthy</h2> <div style="text-align: right;"> <picture> <source srcset="/blog/images/aristotle_300x344.webp" type="image/webp"> <source srcset="/blog/images/aristotle_300x344.png" type="image/png"> <img src="/blog/images/aristotle_300x344.png" title="Mattel Aristotle" class="right liImg" style="width: 300px" alt="Mattel Aristotle" /> </picture> </div> <p> <a href='https://www.engadget.com/2017/01/03/mattel-aristotle-echo-speaker-kids/' target="_blank" rel='nofollow'> Mattel&rsquo;s Aristotle</a> is targeted at children. It uses Microsoft Bing for searching, Microsoft Cortana for voice processing and streams video to the cloud. It can read bedtime stories and play soothing sounds if your child wakes up at the night. It can recognize your children&rsquo;s imperfect speech, and apparently adapts as they get older and become curious about the world. Aristotle is an AI to help raise your child. Did they do a terrific job or a terrible job? It depends on the factors you take into consideration. </p> <p> Aristotle can respond to adults differently than to children. Its logging capability is profound. It tracks things like wet diapers and feedings, and can order supplies when asked by an adult. </p> <p> Aristotle can use object recognition to identify flashcards, or co-opt a toy without electronics and thereby enhance it with sound effects or even another personality. </p> <p> Mattel has about 500 partners, and they have been invited to build connected toys and apps. The hardware uses 256-bit encryption for all transmissions to Aristotle&rsquo;s servers, and data is handled internally in compliance with COPAA and HIPAA. </p> <p class='pullQuote'>It is not difficult to write code that detects when a child is alone</p> <p> How hard would it be to write code that detects when the child is alone? If a malicious entity wanted to, they could embed their program in the device, use IP geolocation and other characteristics to identify specific families, and cause the device to behave differently and/or tell the child things when no adults were paying attention. Who knows what malicious software might do? </p> <h2 id="open">Open Source To The Rescue</h2> <div style="text-align: right;"> <picture> <source srcset="/assets/images/louis.webp" type="image/webp"> <source srcset="/assets/images/louis.png" type="image/png"> <img src="/assets/images/louis.png" title="Justice Louis Brandeis" class="right liImg2 rounded shadow" alt="Justice Louis Brandeis" /> </picture> </div> <p> Closed source systems cannot be adequately vetted for public usage. It is conceivable that selected children might become secretly radicalized by their toy. In 1913 <a href='https://www.brandeis.edu/legacyfund/bio.html' target="_blank" rel='nofollow'>Justice Louis Brandeis</a> said &ldquo;Sunlight is the best disinfectant&rdquo;. Open-source applications, with updates that can be vetted by any interested party, are the only way to ensure these devices are truly safe. </p> Hands-Free Voice as a User Interface 2017-01-10T00:00:00-05:00 https://mslinn.github.io/blog/2017/01/10/voice-activated-apps <p> Amazon&rsquo;s Alexa is a runaway success. Already, 5% of US households have a hands-free voice operated device. 50% of Echo owners keep one in the kitchen. <a href='https://www.gartner.com/doc/3021226/market-trends-voice-ui-consumer' target="_blank" rel='nofollow'>Gartner predicts</a> that by 2018, 30% of our interactions with technology will be via voice conversations. The near future will have many hands-free, voice-driven, Internet-aware applications, and some will also provide a web interface in order to satisfy use cases that require a text or graphic display. </p> <h2 id="near">The Near Future</h2> <p> The near future will have many hands-free, voice-driven, Internet-aware applications, and some will also provide a web interface to satisfy use cases that require a text or graphic display. Potential usages include hands-free voice control of applications running on computers, tablets and phones, hands-free voice control of home devices and vehicles, dictaphones, and games that provide anthropomorphic characters with the ability to generate and understand speech. </p> <div> <div style=""> <picture> <source srcset="/blog/images/google-home_384x500.webp" type="image/webp"> <source srcset="/blog/images/google-home_384x500.png" type="image/png"> <img src="/blog/images/google-home_384x500.png" title="Google Home smart speaker" class=" liImg right rounded shadow" style="height: 500px" alt="Google Home smart speaker" /> </picture> </div> <div style=""> <picture> <source srcset="/blog/images/echo_205x500.webp" type="image/webp"> <source srcset="/blog/images/echo_205x500.png" type="image/png"> <img src="/blog/images/echo_205x500.png" title="Amazon Echo smart speaker" class=" liImg left" style="height: 500px" alt="Amazon Echo smart speaker" /> </picture> </div> </div> <p style="clear: both"> Services that developers and integrators can use for hands-free, voice-operated applications include <a href='https://www.amazon.com/gp/help/customer/display.html?nodeId=201602040' target="_blank" rel='nofollow'>Amazon's Alexa</a>, Google's Assist and Voice, Apple's Siri, and Microsoft's Cortana. </p> <h2 id="present">The Present</h2> <p> Only recently has it become feasible to use hands-free voice as a user interface. The best hands-free, voice applications are innately distributed &mdash; that is, they perform some computation on a local device, and they also require a connection to a server to do the heavy computation necessary for a high-quality experience. Most companies developing applications that use hands-free voice as a user interface require and will continue to require services provided by third parties. However, from the point of view of the application developer's company, the data shared with these third parties can represent a significant security risk. From the user's point of view, this data can represent a potentially significant privacy breach. I'm going to discuss why this is so in this article, and what can be done about it. </p> <p> Privacy and security concerns for synthesizing speech are minimal because quality speech can be synthesized without context specific to an individual. This means that user data need not be associated with the words or phrases being synthesized, so anonymous phrases can be and should be sent to the remote service that generates the audio files produced by the speech synthesis. </p> <p> The best value for high-quality voice generation is achieved with a distributed solution. In this scenario, voice generation is simple to initiate: a text string is sent to a remote service, and an audio clip containing synthesized speech is returned. Most voice generators support embedded markup to control inflection. For example, <a href='https://docs.aws.amazon.com/polly/latest/dg/ssml.html' target="_blank" rel='nofollow'>Amazon Polly</a> and <a href='https://api.ai' target="_blank" rel='nofollow'>Google's Assistant</a> (formerly <code>api.ai</code>) both use <a href='https://developers.google.com/actions/reference/ssml' target="_blank" rel='nofollow'>SSML</a>; Apple uses a <a href='https://eclecticlight.co/2015/12/09/opening-access-text-speech/' target="_blank" rel='nofollow'>variety of techniques</a> across its products, and Microsoft uses <a href='https://en.wikipedia.org/wiki/Microsoft_Speech_API' target="_blank" rel='nofollow'>SAPI</a>. </p> <p> In contrast, the heavy lifting necessary to recognize unconstrained vocabulary requires lots of compute resource, and raises privacy and ethical issues. The main issues are: </p> <ol> <li><a href='#trigger'>Recognizing</a> a trigger word or phrase</li> <li><a href='#training'>Training</a> a voice recognition engine</li> <li>Determining the appropriate <a href='#privacy'>privacy/effectiveness</a> trade-off for your application</li> <li><a href='#integrating'>Integrating</a> with third-party or proprietary services</li> </ol> <h2 id="components">What Are These Things Made From?</h2> <p> Hands-free voice operated devices have most of the same components as a tablet, but often do not have a screen. An ARM A8 to A11 processor is typically embedded in a chip die that also contains a powerful digital signal processor. In other words, they can do a lot of computing; they are more powerful than any mobile phone, and they are more powerful than most tablets. </p> <div class="centered"> <div style=""> <picture> <source srcset="/blog/images/echoParts.webp" type="image/webp"> <source srcset="/blog/images/echoParts.png" type="image/png"> <img src="/blog/images/echoParts.png" title="Amazon Echo parts" class=" liImg" style="width: 100%; max-height: 45%" alt="Amazon Echo parts" /> </picture> </div> <div style=""> <picture> <source srcset="/blog/images/echoBoard.webp" type="image/webp"> <source srcset="/blog/images/echoBoard.png" type="image/png"> <img src="/blog/images/echoBoard.png" title="Amazon Echo PC board" class=" liImg2 rounded shadow" style="width: 100%; max-height: 45%" alt="Amazon Echo PC board" /> </picture> </div> <div style=""> <picture> <source srcset="/blog/images/tiDM37x.webp" type="image/webp"> <source srcset="/blog/images/tiDM37x.png" type="image/png"> <img src="/blog/images/tiDM37x.png" title="TI DM37x processors" class=" liImg2 rounded shadow" style="width: 500px; max-height: 45%" alt="TI DM37x processors" /> </picture> </div> </div> <p style="clear: both"> The open source ecosystem that has grown up around these DSP/CPU chips includes a variety of operatoring systems, including real-time and various Linux distributions, and many applications. The OSes are: Android, DSP/BIOS, Neutrino, Integrity, Windows Embedded CE, Linux, and VxWorks. It is quick and easy to design a complete device similar to Alexa and bring it to market. </p> <h2 id="trigger" style="clear: both">Recognizing a trigger word or phrase</h2> <p> One of the first major challenges one enounters when designing a device that responds to voice, is how to ignore silence, or recognizing noise that should be ignored. This requires some level of signal processing. In contrast, requiring the user to push a button makes for a a much simpler user interface, but this requirement would greatly restrict the types of applications possible. </p> <div style=""> <picture> <source srcset="/blog/images/Icom-IC-706MKIIG_300x180.webp" type="image/webp"> <source srcset="/blog/images/Icom-IC-706MKIIG_300x180.png" type="image/png"> <img src="/blog/images/Icom-IC-706MKIIG_300x180.png" title="Icom 706 Mark II g" class=" right" style="width: 300px" alt="Icom 706 Mark II g" /> </picture> </div> <p> I am a ham radio operator, and I use the well-known protocol for addressing a specific individual when using a broadcast medium. If I want to talk to another ham operator, I address them by their call sign three times and await their acknowledgement before giving my message. &ldquo;KG6LBG, KG6LBG, KG6LBG, this is KG6LDE, do you read me?&rdquo; If that person hears their call sign, they reply &ldquo;KG6LDE, this is KG6LBG, go ahead.&rdquo; &ldquo;KG6LBG, I just called to say hello, over.&rdquo; &ldquo;KG6LDE, Hello yourself, over and out.&rdquo; </p> <p> Hands-free voice applications also need to recognize a trigger word or phrase (also known as a hotword) that prefaces an audio stream which will be processed for voice recognition. Without such a trigger, either the user would need to press a button to start recording their speech, or a continuous audio stream would have to be processed. I'm interested in hands-free voice recognition. Since the voice recognition processing must be done on a remote service, a lot of bandwidth and CPU power would be wasted processing silence or irrelevant sound. Because it is undesirable to a continuous audio stream to a server to accurately recognize trigger words, the methods used to recognize them are less accurate. </p> <h3 id="alternatives">Proprietary Alternatives</h3> <p> Both of these products can be configured to perform the trigger word recognition without requiring any bandwidth between the CPU running the recognition program and a server. Neither of them provide a JavaScript implementation, which means that unless the trigger word recognition program in installed on the local machine, it must be installed on a connected server and bandwidth will be used at all times. </p> <ul> <li> <div style="text-align: right;"> <picture> <source srcset="/blog/images/THF-VC-72dpi.webp" type="image/webp"> <source srcset="/blog/images/THF-VC-72dpi.png" type="image/png"> <img src="/blog/images/THF-VC-72dpi.png" title="Truly Handsfree voice control" class="right " style="width: 250px" alt="Truly Handsfree voice control" /> </picture> </div> Sensory, Inc's <a href='https://www.sensory.com/products/technologies/trulyhandsfree/' target="_blank" rel="nofollow">TrulyHandsfree library</a> (<a href='https://github.com/Sensory/alexa-rpi/blob/master/LICENSE.txt' target="_blank" rel="nofollow">license</a>), based in Santa Clara, CA. &ldquo;We do not have any low cost or free license or library. We are unable to support any student or individual.&rdquo; </li> <li style='clear: both'> <div style="text-align: right;"> <picture> <source srcset="/blog/images/snowboyLogo_250x56.webp" type="image/webp"> <source srcset="/blog/images/snowboyLogo_250x56.png" type="image/png"> <img src="/blog/images/snowboyLogo_250x56.png" title="Snowboy Logo" class="right " style="width: 250px; margin-top: 1em" alt="Snowboy Logo" /> </picture> </div> kitt.ai's <a href='https://github.com/Kitt-AI/snowboy' target="_blank" rel="nofollow">Snowboy</a>, based in Seattle, WA and <a href='https://www.geekwire.com/2016/backed-amazon-paul-allen-kitt-ai-launches-first-hotword-detection-software-toolkit/' target="_blank" rel="nofollow">partially funded by Amazon</a>. </li> </ul> <h3 id="sphinx" style='clear: both'>CMU Sphinx</h3> <div style="text-align: right;"> <picture> <source srcset="/blog/images/CMUSphinx_300x72.webp" type="image/webp"> <source srcset="/blog/images/CMUSphinx_300x72.png" type="image/png"> <img src="/blog/images/CMUSphinx_300x72.png" title="CMU Sphinx Logo" class="right " style="width: 300px" alt="CMU Sphinx Logo" /> </picture> </div> <p> Designed for low-resource platforms, implementations of <a href='http://cmusphinx.sourceforge.net/' target="_blank" rel="nofollow">CMU's Sphinx</a> exist for C (which supports Python) and Java. <a href='http://cmusphinx.sourceforge.net/wiki/faq#qhow_to_implement_hot_word_listening' target="_blank" rel="nofollow">Hotword</a> spotting is supported. <a href='https://en.wikipedia.org/wiki/CMU_Sphinx' target="_blank" rel="nofollow">Several versions</a> of Sphinx exist, with varying free and commercial licenses. Reports suggest that Sphinx works reasonably well but I have not tested yet. Sphinx powers <a href='https://jasperproject.github.io/documentation/faq/' target="_blank" rel="nofollow">Jasper</a>. </p> <h3 id="js">Javascript alternatives</h3> <ul> <li> <a href='https://github.com/TalAter/annyang' target="_blank" rel="nofollow">Annyang</a> works well, but requires Google's web browsers and servers, so it is probably not reasonable to use it just for hotword detection. </li> <li> <a href='https://github.com/jimmybyrum/voice-commands.js' target="_blank" rel="nofollow">Voice-commands.js</a> also requires Google's web browsers and servers. </li> <li> <a href='https://github.com/evancohen/sonus' target="_blank" rel="nofollow">Sonus</a> is a Node framework which will be able to be configured to use a variety of back ends one day. It does hotword detection by using SnowBoy; the authors obviously ignored Snowboy's license terms. </li> <li> <a href='https://github.com/zzmp/juliusjs' target="_blank" rel="nofollow">JuliusJS</a> is a JavaScript port of the &ldquo;Large Vocabulary Continuous Speech Recognition Engine Julius&rdquo;. It does not call any servers and runs in most browsers. Recognition is weak and it requires a lot of CPU. </li> <li <a href='https://justbuildsomething.com/cross-browser-voice-recognition-with-pocketsphinx-js/' target="_blank" rel="nofollow">PocketSphinx.js</a> is a free browser-based alternative, unfortunately it is horrible. </li> </ul> <h2 id="training">Training a voice recognition engine</h2> <p>Voice recognition engines need to be trained on a large dataset for the desired languages. Recognition effectiveness is less than linearly proportional to the size of the dataset, and high-quality datasets are important. Truly large amounts of data are required. Amazon, Apple, Google and Microsoft have commercial products that were trained using enormous proprietary datasets. This is a substantial investment, so only well-capitalized organizations will be able to offer their own voice recognition engines for unconstrained vocabularies. </p> <h2 id="privacy">Determining Your Application's Privacy / Effectiveness Tradeoff</h2> <p> A voice recognition's effectiveness increases for specific users if their voice streams are recorded and stored, then used for further training. However, this means that <a href='https://www.reddit.com/r/technology/comments/2wzmmr/everything_youve_ever_said_to_siricortana_has/' target="_blank" rel="nofollow">privacy</a> and security are traded off for effectiveness. This service provider's tradeoff might not be optimal for your use case. Apple's Siri only associates the stored voice recordings with you for <a href='https://www.wired.com/2013/04/siri-two-years/' target="_blank" rel="nofollow">6 months</a>. Google's Assist and Voice, and <a href='https://www.amazon.com/gp/help/customer/display.html?nodeId=201602040' target="_blank" rel="nofollow">Amazon's Alexa</a> store all your voice recordings forever, unless you explicitly delete them. I could not discover how long Microsoft's Cortana and Skype store voice recordings, or how to delete them. <p> <h2 id="integrating">Integrating With Third-party or Proprietary Services</h2> <p> Because voice recognition returns a structured document like JSON or XML, and voice generation is simple to initiate, integration is well understood and many options exist. </p> Setting Up Jekyll with Ubuntu or WSL 2017-01-08T00:00:00-05:00 https://mslinn.github.io/blog/2017/01/08/setting-up-github-pages <p> Here is a script I wrote in April 2020 to set up a local development environment for this website. The script uses <a href="https://jekyllrb.com/" target="_blank" rel="nofollow">Jekyll</a> to assemble the website that you are currently reading. </p> <pre data-lt-active='false'> #!/bin/bash # Installs the right version of Jekyll and all dependencies function installGem { sudo -H gem install $1 -v $2 --source 'https://rubygems.org/' } yes | sudo apt install bundler make ruby ruby-dev software-properties-common zlib1g-dev sudo -H gem update --system 3.0.6 sudo -H gem uninstall i18n jekyll jekyll-docs jekyll-sass-converter public_suffix rouge --all installGem jekyll-timeago 0.13.1 installGem backports 3.17.1 installGem jekyll 3.3.0 #installGem jekyll-docs 3.3.0 sudo -H gem install bundler classifier-reborn github-pages jekyll-admin jekyll-assets jekyll-docs jekyll-gist jekyll-tagging jekyll-theme-architect html-proofer libz-dev sprockets installGem jekyll-sass-converter 1.5.2 installGem i18n 0.9.5 installGem nokogiri 1.10.9 bundle install </pre> <p> This remainder of this article is obsolete. <a href="https://jekyllrb.com/docs/" target="_blank" rel="nofollow">Follow these instructions instead.</a> </p> <hr /> <p> These are my notes for setting up GitHub pages using Ubuntu or Windows Subsystem for Linux. I updated these notes Jan 3, 2018 to include instructions on <code>jekyll-admin</code>. Since then, Jekyll has continued to evolve and these instructions should no longer be followed. I leave this page merely for posterity's sake. </p> <p> Read the docs on the <a href='https://jekyllrb.com/docs/github-pages/#use-the-github-pages-gem' target="_blank" rel="nofollow"><code>github-pages</code> gem</a>. </p> <p>Make a <code>Gemfile</code> with the following contents:</p> <pre data-lt-active='false'> source "https://rubygems.org" gem "classifier-reborn" gem "github-pages", group: :jekyll_plugins gem "html-proofer" gem "jekyll" gem 'jekyll-admin', group: :jekyll_plugins gem "jekyll-assets" gem "jekyll-docs" gem "jekyll-gist" gem "jekyll-theme-architect" gem "sprockets" </pre> <p> Create <code>_config.yml</code> with the following contents: </p> <pre data-lt-active='false'> exclude: [vendor] jekyll_admin: hidden_links: # - posts # - pages # - staticfiles # - datafiles # - configuration markdown: kramdown name: Mike Slinn, Connoisseur of Technology permalink: /blog/:year/:month/:day/:title plugins: [classifier-reborn, html-proofer, jekyll, jekyll-admin, jekyll-assets, jekyll-docs, jekyll-gist, jekyll-theme-cayman] title: Mike Slinn's Blog </pre> <p> Ruby 2.3+ is required, but Ubuntu defaults to an older version. I set up Ruby 2.3, with the option of installing other versions and making them default. For more background, see <a href='https://www.brightbox.com/docs/ruby/ubuntu/#Addingtherepository' target="_blank" rel="nofollow">Ruby packages for Ubuntu</a>. I also installed various gems necessary to provide the Jekyll functionality I desired. </p> <pre data-lt-active='false'> sudo apt-get install make software-properties-common sudo apt-add-repository ppa:brightbox/ruby-ng sudo apt-get update sudo apt install ruby2.3 ruby2.3-dev ruby-switch zlib1g-dev ruby-bundler ruby-switch --list sudo ruby-switch --set ruby2.3 sudo gem update --system sudo gem install bundler classifier-reborn jekyll github-pages jekyll-assets jekyll-gist \ jekyll-docs jekyll-theme-cayman html-proofer classifier-reborn jekyll-admin sprockets sudo bundle clean --force bundle install </pre> <p> Create repo <code>userId.github.io</code> (where <code>userId</code> is your GitHub user id) and clone it. </p> <pre data-lt-active='false'> <span class="unselectable">$ </span>git clone git@github.com:userId/userId.github.io.git <span class="unselectable">$ </span>cd userId.github.io/</pre> <h2>Running Jekyll</h2> <p>Read the docs.</p> <pre data-lt-active='false'> <span class="unselectable">$ </span>bundle exec jekyll docs</pre> <p> <a href='https://github.com/Microsoft/BashOnWindows/issues/216' target="_blank" rel="nofollow">Read</a> about how Bash on Windows does not yet support watched directories properly. </p> <pre data-lt-active='false'> <span class="unselectable">$ </span>bundle exec jekyll serve --force_polling </pre> <p>For other OSes:</p><pre data-lt-active='false'> <span class="unselectable">$ </span>bundle exec jekyll serve</pre> <p> Use the <code>--drafts</code> option to preview draft blog posts in the <code>_drafts</code> directory. For Bash on Windows: </p> <pre data-lt-active='false'> <span class="unselectable">$ </span>bundle exec jekyll serve --force_polling --drafts</pre> <p>For other OSes:</p> <pre data-lt-active='false'> <span class="unselectable">$ </span>bundle exec jekyll serve --drafts</pre> <h2>Visual Blog Editor</h2> <p><a href="https://github.com/planetjekyll/awesome-jekyll-editors" target="_blank" rel="nofollow">Many awesome Jekyll editors</a> exist. The above instructions installed <a href="https://github.com/jekyll/jekyll-admin" target="_blank" rel="nofollow">jekyll-admin</a>. Run Jekyll as described above and navigate to <a href="http://localhost:4000/admin" target="_blank" rel="nofollow">http://localhost:4000/admin</a> to access the administrative interface. </p> <p> Unfortunately, <code>jekyll-admin</code> does not provide an WYSIWYG editor like that provided by CKEditor. I <a href="https://github.com/jekyll/jekyll-admin/issues/437#issuecomment-355209137" target="_blank" rel="nofollow">suggested this new feature</a>. </p> I Updated the Apache Spark Reference Applications 2016-11-15T00:00:00-05:00 https://mslinn.github.io/blog/2016/11/15/lessons-from-updating-the-twitter-classifier-apache-spark-reference-application <h2 id="intro">Overview</h2> <p> The Apache Spark committers just accepted my pull request that updated the official <a href='https://github.com/databricks/reference-apps/tree/master/twitter_classifier' target="_blank" rel="nofollow">Twitter Classifier Reference Application</a> from Spark 1.4 / Scala 2.10 to Spark 2 / Scala 2.11. This post discusses some things I did in the pull request from the point of view of a Scala programmer. A primary goal was to rewrite the reference application using idiomatic and functional-style Scala. This post briefly discusses two unique aspects that I addressed: command-line parsing and DRYing up the code by importing scopes. I did several other things to improve the reference application, such as modularizing the code and providing run scripts, but this post does not address them because those techniques are generally well understood. </p> <p> I did not upgrade the reference application to Scala 2.12, which was released a couple of weeks ago because Spark does not yet support Scala 2.12. Josh Rosen of Databricks wrote me and said: </p> <blockquote> &ldquo;Some of Spark’s dependencies and by Scala-version-specific code changes necessary to work around method overloads became ambiguous in 2.12. The umbrella ticket tracking 2.12 support can be found at <a href='https://issues.apache.org/jira/browse/SPARK-14220' target="_blank" rel="nofollow"><code>issues.apache.org/jira/browse/SPARK-14220</code></a>. One of the hardest pieces will be <a href='https://issues.apache.org/jira/browse/SPARK-14643' target="_blank" rel="nofollow"><code>issues.apache.org/jira/browse/SPARK-14643</code></a> (see the linked design document on that issue). Lack of 2.12 support for Breeze and its dependencies is likely to be another serious blocker, but that might be avoidable by only publishing a subset of the projects with 2.12 to begin with (e.g. only Spark core / SQL at first).&rdquo; </blockquote> <h2 id="cli">Command Line Parsing</h2> <p>I modified the reference applications’ command line parsing to use a Scala library that supported idiomatic Scala (<a href='https://github.com/acrisci/commander-scala' target="_blank" rel="nofollow">Commander Scala</a>), instead of <a href='https://commons.apache.org/proper/commons-cli/' target="_blank" rel="nofollow">Apache Commons CLI</a>, which is the Java library that was previously used. The result was simple, clean and very terse code that is intuitive to understand and easy to maintain. Commander Scala automatically generates the help message. Take a look at the <a href='https://github.com/databricks/reference-apps/blob/1793e3dc2335696e98a335130673b58b35086c26/twitter_classifier/scala/src/main/scala/com/databricks/apps/twitterClassifier/CollectOptions.scala' target="_blank" rel="nofollow"><code>collect</code> command&rsquo;s parsing</a>. You’ll notice that it uses some common code for parsing Twitter authentication parameters. This code is much shorter than the previous code, easier to understand and modify, and is more flexible.</p> <pre data-lt-active='false'>import com.github.acrisci.commander.Program import java.io.File abstract sealed case class CollectOptions( twitterOptions: TwitterOptions, overWrite: Boolean = false, tweetDirectory: File = new File(System.getProperty("user.home"), "/sparkTwitter/tweets/"), numTweetsToCollect: Int = 100, intervalInSecs: Int = 1, partitionsEachInterval: Int = 1 ) object CollectOptions extends TwitterOptionParser { override val _program = super._program .option(flags="-w, --overWrite", description="Overwrite all data files from a previous run") .usage("Collect [options] &lt;tweetDirectory> &lt;numTweetsToCollect> &lt;intervalInSeconds> &lt;partitionsEachInterval>") def parse(args: Array[String]): CollectOptions = { val program: Program = _program.parse(args) if (program.args.length!=program.usage.split(" ").length-2) program.help new CollectOptions( twitterOptions = super.apply(args), overWrite = program.overWrite, tweetDirectory = new File(program.args.head.replaceAll("^~", System.getProperty("user.home"))), numTweetsToCollect = program.args(1).toInt, intervalInSecs = program.args(2).toInt, partitionsEachInterval = program.args(3).toInt ){} } }</pre> <p> Here is how to sidestep the Spark help message and display the help message for the collect entry point: </p> <pre data-lt-active='false'>$ <b>spark-shell \ -class com.databricks.apps.twitterClassifier.Collect \ -jars target/scala-2.11/spark-twitter-lang-classifier-assembly-2.0.0.jar \ -- -help</b> Usage: Collect [options] &lt;tweetDirectory> &lt;numTweetsToCollect> &lt;intervalInSeconds> &lt;partitionsEachInterval> Options: -h, — help output usage information -V, — version output the version number -w, — overWrite Overwrite all data files from a previous run -v, — accessTokenSecret [type] Twitter OAuth Access Token Secret -t, — accessToken [type] Twitter OAuth Access Token -s, — consumerSecret [type] Twitter OAuth Consumer Secret -c, — consumerKey [type] Twitter OAuth Consumer Key</pre> <h2 id="import">Importing Inner Scope Into Another Object</h2> <p> Apache Spark is unusual in that you cannot encapsulate a Spark streaming context in a type instance. A memory overflow occurs when you try to instantiate a Scala trait or class that creates a Spark context. The solution is to use a unique Scala feature: the ability to import inner scope from an object into another scope. This meant that the code was made DRY (common code was not repeated), without using classes or traits. </p> <p> Here is how I took advantage of this little-known Scala technique: first I defined the <a href='https://github.com/databricks/reference-apps/blob/1793e3dc2335696e98a335130673b58b35086c26/twitter_classifier/scala/src/main/scala/com/databricks/apps/twitterClassifier/package.scala#L7-L19' target="_blank" rel="nofollow"><code>SparkObject</code> object</a> within a package object so it was easily found: </p> <pre data-lt-active='false'>object SparkSetup { val spark = SparkSession .builder .appName(getClass.getSimpleName.replace("$", "")) .getOrCreate() val sqlContext = spark.sqlContext val sc: SparkContext = spark.sparkContext sc.setLogLevel("ERROR") }</pre> <p> Next I imported all the variables defined in <code>SparkSetup</code> into the <code>Collect</code> object’s scope, including <code>sc</code>, which was used twice, <a href='https://github.com/databricks/reference-apps/blob/1793e3dc2335696e98a335130673b58b35086c26/twitter_classifier/scala/src/main/scala/com/databricks/apps/twitterClassifier/Collect.scala' target="_blank" rel="nofollow">like this</a>: </p> <pre data-lt-active='false'>object Collect extends App { val options = CollectOptions.parse(args) import SparkSetup._ val ssc = new StreamingContext(sc, Seconds(options.intervalInSecs)) Collector.doIt(options, sc, ssc) }</pre> <p> Want to learn more practical Scala techniques? Head over to <a href='https://www.ScalaCourses.com' target="_blank" rel="nofollow"><code>ScalaCourses.com</code></a> and enroll! The combination of the Introduction to Scala and Intermediate Scala courses will teach you everything you need to know to start your journey with Apache Spark. </p> <p> Mike Slinn is the lead Scala instructor at <a href="https://www.ScalaCourses.com" target="_blank" rel="nofollow">ScalaCourses.com</a>. </p> Publishing Maven Artifacts to AWS S3 2013-07-07T00:00:00-04:00 https://mslinn.github.io/blog/2013/07/07/publishing-maven-artifacts-to-aws-s3 <p> I wanted a simple, flexible and cheap way of publishing proprietary Maven artifacts created by <code>sbt</code> projects such that they could be securely retrieved by authorized individuals. I liked the idea of versioned artifacts, but did not want to use GitHub or BitBucket to host the artifacts because of the hassle of maintaining ever-larger git repos. Instead, I opted for S3's optional versioning mechanism. </p> <p> The technique described here relies on the fact that publishing to a local file (using <code>sbt publish</code>) generates all the necessary artifacts, which merely need to be copied to the right directory on the Artifactory server. The server need not be anything special: a normal web server works fine, as does <code>webdav</code>. A variety of other protocols are also supported by <code>sbt</code>. </p> <p> I created two test projects on GitHub: <code><a href="https://github.com/mslinn/testPublishLib" target="_blank" rel="nofollow">testPublishLib</a></code> and <code><a href="https://github.com/mslinn/testPublishApp" target="_blank" rel="nofollow">testPublishApp</a></code>. You could clone them if you would like to try this. Each of them has a <code>README</code> that explains what they do.</p> I used <code>s3cmd</code> to manage the AWS S3 buckets that hold the repository. You can <a href="https://s3tools.org/s3cmd" target="_blank" rel="nofollow">obtain <code>s3cmd</code></a> for most OSes. I found I needed to install <code>python-magic</code>: </p> <pre data-lt-active='false'><span class="unselectable">$ </span>sudo pip install python-magic</pre> <ol> <li>Let <code>s3cmd</code> know your s3 keys: <pre data-lt-active='false'><span class="unselectable">$ </span>s3cmd --configure</pre> </li> <li>Create the S3 bucket, which must be unique. If you want to repository to be publicly visible, be sure that the bucket name starts with <code>www.</code> If you already have the S3 bucket then just omit this step. <pre data-lt-active='false'><span class="unselectable">$ </span>s3cmd mb s3://www.mymavenrepo</pre> </li> <li>If you want to repository to be publicly visible, you need to enable the S3 bucket web site option: <pre data-lt-active='false'><span class="unselectable">$ </span>s3cmd ws-create s3://www.mymavenrepo</pre></li> <li>Publish your library to a local repository. <a href="https://github.com/mslinn/testPublishLib" target="_blank" rel="nofollow"><code>testPublishLib</code></a> is an example of how to set up a project properly. <pre data-lt-active='false'><span class="unselectable">$ </span>sbt publish</pre></li> <li>Copy your locally published artifacts to S3. You can either type it out longhand: <pre data-lt-active='false'><span class="unselectable">$ </span>s3cmd -P sync ~/.ivy2/local/com/micronautics/test_publish_lib \ s3://www.mymavenrepo/snapshots/com/micronautics/test_publish_lib</pre> ... (note that the <code>-P</code> option makes the files publicly visible), or you can use <code>s3publish</code>: <pre data-lt-active='false'><span class="unselectable">$ </span>s3publish com/micronautics/test_publish_lib</pre> </li> <li>If your repository is not public, you will need to provide authentication in a file which I called <tt>~/.sbt/awsCreds.sbt</tt> for convenience: <pre data-lt-active='false'>credentials += Credentials("AWS Realm", "www.mavenrepo.s3.amazonaws.com", "myUserId", "myPassword")</pre> </li> <li>Use the published artifact from your sbt project by including a resolver of the form: <pre data-lt-active='false'>"AWS Snapshots" at "https://www.mavenrepo.s3.amazonaws.com/snapshots"</pre> The <code>TestPublishApp</code> project is a working example of how to do that.</li> </ol> <p> Following is the script I wrote to upload to S3, which I called <code>s3publish</code>. It assumes you are always publishing a snapshot; and that the files should be public. I leave it to you to extend this script to handle releases and private content if you have the need. </p> <noscript><pre>#!/bin/bash if [ $# -eq 0 ]; then echo &quot;Usage: `basename $0` pubpath/name [repo]&quot; echo &quot; Where pubpath might be something like com/micronautics&quot; echo &quot; name is name of artifact to publish&quot; echo &quot; repo is the optional name of the bucket to publish to&quot; echo &quot; Example: `basename $0` com/micronautics/test_publish_lib&quot; exit 1 fi REPO=www.mymavenrepo OPTIONS=-P if [ $# -eq 2 ]; REPO=$2; fi s3cmd $OPTIONS sync ~/.ivy2/local/$1 s3://$REPO/snapshots/$1</pre></noscript><script src="https://gist.github.com/mslinn/5945115.js"> </script> Load Testing ScalaCourses.com 2013-06-01T00:00:00-04:00 https://mslinn.github.io/blog/2013/06/01/load-testing-scalacoursescom <div style="text-align: center;"> <picture> <source srcset="/assets/images/ScalaCoursesEclipse.webp" type="image/webp"> <source srcset="/assets/images/ScalaCoursesEclipse.png" type="image/png"> <img src="/assets/images/ScalaCoursesEclipse.png" title="ScalaCourses logo" class="center liImg2 rounded shadow" alt="ScalaCourses logo" /> </picture> </div> <p> <a href="https://scalacourses.com/" target="_blank" rel="nofollow">ScalaCourses.com</a>, which will be announced next week, is built using the entire Typesafe stack: Scala 2.10, Play 2.1, Slick 1.0 and Akka 2.1. It runs on Heroku. </p> <p> I ran a load test on the app running on only one Heroku dyno. I configured JMeter to use 300 threads, hammering at full speed (no pauses between hits). Testing was done from my desktop. The test generated 16.5Mb/s inbound and 4.1Mb/s outbound according to <code>iptraf</code>. The test did not download page assets because they are served directly from AWS S3. </p> <p> 50% of all responses were received in under 110ms, and 95% were received in under 165ms. The distance from my workstation in Half Moon Bay, CA, USA to the Heroku app server, running from an Amazon server in Ashburn, Virginia, USA is about 3000 miles or 4,828 km away. Considering that average ping time is ~100ms, that is amazingly good! Ping measures round-trim time for a test packet, and <code>mtr</code> showed the average ping time as ~95ms with a standard deviation of 18. </p> <div style=""> <picture> <source srcset="/blog/images/jmeter_690x448.webp" type="image/webp"> <source srcset="/blog/images/jmeter_690x448.png" type="image/png"> <img src="/blog/images/jmeter_690x448.png" title="50% of all responses were received in under 110ms, and 95% were received in under 165ms" class=" liImg2 rounded shadow" alt="50% of all responses were received in under 110ms, and 95% were received in under 165ms" /> </picture> </div> <p> Yes, I did put a lot of care into the design of the app so that it would scale well, but I had not expected such fantastic results. Kudos to each of the Typesafe product teams, and to Heroku! </p> Cleaning the Heroku Cache 2013-03-18T00:00:00-04:00 https://mslinn.github.io/blog/2013/03/18/cleaning-heroku-cache <p> In an <a href="/blog/2013/02/27/command-line-sbt-on-heroku-dyno.html">earlier post</a>, I talked about using a bash console to experiment with a Heroku app. I mentioned that you should not run <code>sbt</code>. Of course, that is exactly what I did, and I discovered the hard way that the cache can't be cleared from the Heroku bash shell. The build cache is held outside the dyno and cannot be accessed from inside a running dyno. </p> <p> Cleaning the cache is accomplished with a special cache cleaner buildpack. Set the buildpack in your app like this: </p> <pre data-lt-active='false'>heroku config:add BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-scala.git#cleancache --app myapp</pre> <p>Push your code:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>git push heroku master</pre> <p>Remove the cache cleaner buildpack:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>heroku config:remove BUILDPACK_URL --app myapp</pre> <p>Change a file, commit and push again:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>git push heroku master</pre> <p>All better!</p> Using Scala’s String Interpolation to Access a Map 2013-03-15T00:00:00-04:00 https://mslinn.github.io/blog/2013/03/15/using-scala-210s-string-interpolation <p> Given a map, would it not be nice to have a shorthand way of looking up a value from a key, and to provide a default value? I&rsquo;ve wanted to be able to do this for a long time. Scala 2.10 makes this really easy! </p> <p> The <code>MapLookup</code> class below contains a <code>Map[String, Int]</code> that can be looked up by using a dollar sign ($) from code that contains a reference to the implicit class. If the lookup key is not defined by the map, a zero is returned. </p> <pre data-lt-active='false'>implicit class MapLookup(val sc: StringContext) { val map = Map(("a", 1), ("b", 2), ("c", 3)).withDefaultValue(0) def $(args: Any*): Int = { val orig = sc.s (args : _*) map.get(orig) } }</pre> <p> Assuming that the above is stored in a file called <code>strInterp.scala</code>, here are some examples of usage: </p> <pre data-lt-active='false'><span class="unselectable">$ </span>scala -i strInterp.scala Loading strInterp.scala... defined class MapLookup Welcome to Scala version 2.10.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_17). Type in expressions to have them evaluated. Type :help for more information. scala&gt; $"a" res2: Int = 1 scala&gt; $"z" res3: Int = 0</pre> <p>Short and sweet!</p> Listing of all AWS Elastic Transcoder Presets 2013-03-15T00:00:00-04:00 https://mslinn.github.io/blog/2013/03/15/listing-of-all-aws-elastic-transcoder <p> Here is a listing of all <a href="https://console.aws.amazon.com/elastictranscoder/home?region=us-east-1#presets:" target="_blank" rel="nofollow">AWS Elastic Transcoder presets</a>, provided &lsquo;out of the box&rsquo; as system-wide presets. </p> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-000001</code><br> <b>Name:</b> <code>System preset: Generic 1080p</code><br> <b>Description:</b> <code>System preset generic 1080p</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=baseline, Level=4},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 5400,FrameRate: 29.97,AspectRatio: 16:9,MaxWidth: 1920,MaxHeight: 1080,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-000010</code><br> <b>Name:</b> <code>System preset: Generic 720p</code><br> <b>Description:</b> <code>System preset generic 720p</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=baseline, Level=3.1},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 2400,FrameRate: 29.97,MaxWidth: 1280,MaxHeight: 720,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-000020</code><br> <b>Name:</b> <code>System preset: Generic 480p 16:9</code><br> <b>Description:</b> <code>System preset generic 480p 16:9</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 128,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=baseline, Level=3.1},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 1200,FrameRate: 29.97,MaxWidth: 854,MaxHeight: 480,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-000030</code><br> <b>Name:</b> <code>System preset: Generic 480p 4:3</code><br> <b>Description:</b> <code>System preset generic 480p 4:3</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 128,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=baseline, Level=3},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 900,FrameRate: 29.97,MaxWidth: 640,MaxHeight: 480,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-000040</code><br> <b>Name:</b> <code>System preset: Generic 360p 16:9</code><br> <b>Description:</b> <code>System preset generic 360p 16:9</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 128,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=baseline, Level=3},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 720,FrameRate: 29.97,MaxWidth: 640,MaxHeight: 360,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-000050</code><br> <b>Name:</b> <code>System preset: Generic 360p 4:3</code><br> <b>Description:</b> <code>System preset generic 360p 4:3</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 128,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=baseline, Level=3},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 600,FrameRate: 29.97,MaxWidth: 480,MaxHeight: 360,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-000060</code><br> <b>Name:</b> <code>System preset: Generic 320x240</code><br> <b>Description:</b> <code>System preset generic 320x240</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 22050,BitRate: 64,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=baseline, Level=1.3},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 300,FrameRate: 15,MaxWidth: 320,MaxHeight: 240,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-100010</code><br> <b>Name:</b> <code>System preset: iPhone4</code><br> <b>Description:</b> <code>System preset: iPod touch 5G, 4G, iPad 1G, 2G</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 48000,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=main, Level=3.1},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 2200,FrameRate: 30,MaxWidth: 1280,MaxHeight: 720,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-100020</code><br> <b>Name:</b> <code>System preset: iPhone4S</code><br> <b>Description:</b> <code>System preset: iPhone 5, iPad 3G, 4G, iPad mini, Samsung Galaxy S2/S3/Tab 2</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 48000,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=high, Level=4.1},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 5000,FrameRate: 30,MaxWidth: 1920,MaxHeight: 1080,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-100030</code><br> <b>Name:</b> <code>System preset: iPhone3GS</code><br> <b>Description:</b> <code>System preset: iPhone 3GS</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 48000,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=baseline, Level=3},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 600,FrameRate: 30,MaxWidth: 640,MaxHeight: 480,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-100040</code><br> <b>Name:</b> <code>System preset: iPod Touch</code><br> <b>Description:</b> <code>System preset: iPhone 1, 3, iPod classic</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 48000,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=baseline, Level=3},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 1500,FrameRate: 30,MaxWidth: 640,MaxHeight: 480,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-100050</code><br> <b>Name:</b> <code>System preset: Apple TV 2G</code><br> <b>Description:</b> <code>System preset: Apple TV 2G</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 48000,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=main, Level=3.1},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 5000,FrameRate: 30,MaxWidth: 1280,MaxHeight: 720,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-100060</code><br> <b>Name:</b> <code>System preset: Apple TV 3G</code><br> <b>Description:</b> <code>System preset: Apple TV 3G, Roku HD/2 XD</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 48000,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=high, Level=4},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 5000,FrameRate: 30,MaxWidth: 1920,MaxHeight: 1080,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-100070</code><br> <b>Name:</b> <code>System preset: Web</code><br> <b>Description:</b> <code>System preset: Facebook, SmugMug, Vimeo, YouTube</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=main, Level=3.1},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 2200,FrameRate: 30,MaxWidth: 1280,MaxHeight: 720,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-100080</code><br> <b>Name:</b> <code>System preset: KindleFireHD</code><br> <b>Description:</b> <code>System preset: Kindle Fire HD</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 48000,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=main, Level=4},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 2200,FrameRate: 30,MaxWidth: 1280,MaxHeight: 720,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-100090</code><br> <b>Name:</b> <code>System preset: KindleFireHD8.9</code><br> <b>Description:</b> <code>System preset: Kindle Fire HD 8.9</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 48000,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=main, Level=4},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 5400,FrameRate: 30,MaxWidth: 1920,MaxHeight: 1080,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-100100</code><br> <b>Name:</b> <code>System preset: KindleFire</code><br> <b>Description:</b> <code>System preset: Kindle Fire</code><br> <b>Container:</b> <code>mp4</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 48000,BitRate: 160,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=main, Level=3.1},KeyframesMaxDist: 90,FixedGOP: false,BitRate: 1600,FrameRate: 30,MaxWidth: 1024,MaxHeight: 576,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-200010</code><br> <b>Name:</b> <code>System preset: HLS 2M</code><br> <b>Description:</b> <code>System preset: HLS 2M</code><br> <b>Container:</b> <code>ts</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 128,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=main, Level=3.1},KeyframesMaxDist: 90,FixedGOP: true,BitRate: 1872,FrameRate: auto,MaxWidth: 1024,MaxHeight: 768,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-200020</code><br> <b>Name:</b> <code>System preset: HLS 1.5M</code><br> <b>Description:</b> <code>System preset: HLS 1.5M</code><br> <b>Container:</b> <code>ts</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 128,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=main, Level=3.1},KeyframesMaxDist: 90,FixedGOP: true,BitRate: 1372,FrameRate: auto,MaxWidth: 960,MaxHeight: 640,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-200030</code><br> <b>Name:</b> <code>System preset: HLS 1M</code><br> <b>Description:</b> <code>System preset: HLS 1M</code><br> <b>Container:</b> <code>ts</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 128,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=main, Level=3.1},KeyframesMaxDist: 90,FixedGOP: true,BitRate: 872,FrameRate: auto,MaxWidth: 640,MaxHeight: 432,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-200040</code><br> <b>Name:</b> <code>System preset: HLS 600k</code><br> <b>Description:</b> <code>System preset: HLS 600k</code><br> <b>Container:</b> <code>ts</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 128,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=3, Profile=baseline, Level=3.0},KeyframesMaxDist: 90,FixedGOP: true,BitRate: 472,FrameRate: auto,MaxWidth: 480,MaxHeight: 320,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> <div style="margin-bottom: 8pt;"> <b>Id:</b> <code>1351620000001-200050</code><br> <b>Name:</b> <code>System preset: HLS 400k</code><br> <b>Description:</b> <code>System preset: HLS 400k</code><br> <b>Container:</b> <code>ts</code><br> <b>Audio:</b> <code>{Codec: AAC,SampleRate: 44100,BitRate: 128,Channels: 2}</code><br> <b>Video:</b> <code>{Codec: H.264,CodecOptions: {MaxReferenceFrames=1, Profile=baseline, Level=3.0},KeyframesMaxDist: 90,FixedGOP: true,BitRate: 272,FrameRate: auto,MaxWidth: 400,MaxHeight: 288,DisplayAspectRatio: auto,SizingPolicy: ShrinkToFit,PaddingPolicy: NoPad}</code> </div> Bash shell on a Heroku Dyno 2013-02-27T00:00:00-05:00 https://mslinn.github.io/blog/2013/02/27/command-line-sbt-on-heroku-dyno <p> Up to now most of my work with Heroku has been via the <code>heroku</code> command line client, and pushing builds to app instances to have them compiled and run. Recently, I've been messing around with the remote <code>bash</code> shell of my Heroku apps. To enter a remote shell, just type the following from the root directory of a git project that declares your Heroku app as a remote repository. This command uses the <code>heroku</code> client provided by the Heroku toolbelt: </p> <pre data-lt-active='false'><span class="unselectable">$ </span>heroku run bash</pre> <p> Upon login, you are placed into the root directory of your deployed app. Unfortunately, you do not have access to the table of mounted file systems and you cannot run <code>sudo</code>. </p> <pre data-lt-active='false'><span class="unselectable">~ $ </span>pwd /app <span class="unselectable">~ $ </span>ls -alF total 64 drwx------ 10 u37570 37570 4096 Dec&nbsp; 1 01:07 ./ drwxr-xr-x 15 root&nbsp;&nbsp; root&nbsp; 4096 Oct 31&nbsp; 2011 ../ -rwx------&nbsp; 1 u37570 37570&nbsp; 183 Dec&nbsp; 1 01:05 .gitignore* drwx------&nbsp; 3 u37570 37570 4096 Dec&nbsp; 1 01:05 .ivy2/ drwxrwxr-x&nbsp; 6 u37570 37570 4096 Dec&nbsp; 1 01:05 .jdk/ drwx------&nbsp; 4 u37570 37570 4096 Dec&nbsp; 1 01:08 .sbt_home/ -rw-------&nbsp; 1 u37570 37570&nbsp;&nbsp; 98 Dec&nbsp; 1 01:05 Procfile -rw-------&nbsp; 1 u37570 37570 3833 Dec&nbsp; 1 01:05 README.md drwx------&nbsp; 5 u37570 37570 4096 Dec&nbsp; 1 01:05 app/ drwx------&nbsp; 2 u37570 37570 4096 Dec&nbsp; 1 01:05 conf/ drwx------&nbsp; 5 u37570 37570 4096 Dec&nbsp; 1 01:06 project/ drwx------&nbsp; 5 u37570 37570 4096 Dec&nbsp; 1 01:05 public/ -rwx------&nbsp; 1 u37570 37570&nbsp;&nbsp; 27 Dec&nbsp; 1 01:05 system.properties* drwx------&nbsp; 6 u37570 37570 4096 Dec&nbsp; 1 01:08 target/</pre> <p>Discover total used file space (my slug uses 1.7 GB):</p> <pre data-lt-active='false'><span class="unselectable">~ $ </span>du -sh / du: cannot read directory `/proc/tty/driver': Permission denied du: cannot read directory `/proc/1/task/1/fd': Permission denied du: cannot read directory `/proc/1/task/1/fdinfo': Permission denied du: cannot read directory `/proc/1/fd': Permission denied du: cannot read directory `/proc/1/fdinfo': Permission denied du: cannot access `/proc/11/task/11/fd/4': No such file or directory du: cannot access `/proc/11/task/11/fdinfo/4': No such file or directory du: cannot access `/proc/11/fd/4': No such file or directory du: cannot access `/proc/11/fdinfo/4': No such file or directory du: cannot read directory `/lost+found': Permission denied du: cannot read directory `/etc/ssl/private': Permission denied 1.7G /</pre> <p>All your environment variables are available:</p> <pre data-lt-active='false'><span class="unselectable">~ $ </span>set # Miles of output</pre> <p><code>ifconfig</code> is not available, but the IP address of your dyno can be discovered this way:</p> <pre data-lt-active='false'><span class="unselectable">~ $ </span>tail -n 1 /etc/hosts 10.92.81.76 e3d315e6-4392-4f3a-b7ae-75438c469697</pre> <p> You can run <code>sbt</code>, but this is a bad idea because the Ivy cache is held outside the dyno, and there is no way for you to clean it without using some voodoo. </p> <pre data-lt-active='false'><span class="unselectable">~ $ </span>sbt update [info] Loading global plugins from /app/.sbt_home/.sbt/plugins [info] Loading project definition from /app/project [info] Set current project to blahblah (in build file:/app/) [info] Updating {file:/app/}blahblah... ... miles of output ... [info] Done updating. [success] Total time: 6 s, completed Feb 28, 2013 4:40:21 AM</pre> AWS S3 websites and Naked HTTP Redirects 2012-11-14T00:00:00-05:00 https://mslinn.github.io/blog/2012/11/14/aws-s3-web-sites-and-naked-http <p> Sites hosted directly off AWS S3 only respond to the <tt>www</tt> subdomain. In other words, if you tried to navigate to <tt>https://mysite.com</tt> for a site that was hosted on AWS S3, a 404 status would result, however <tt>https://www.mysite.com</tt> would work. Some registrars, like Namecheap and GoDaddy have URL Redirect, while others, like GKG.net, do not. </p> <p> Here is how to set up Namecheap to redirect requests like <tt>https://mysite.com/blah</tt> to <tt>https://www.mysite.com/blah</tt> so AWS S3 will respond. I also show how to set up the DNS for email with Rackspace. </p> <ol> <li>Originally AWS buckets could only be used for serving websites if they started with <tt>www</tt>, for example: <code>www.artforhealingenvironments</code>. This restriction no longer exists. </li> <li>Go to the <a href="https://www.namecheap.com/products/freedns.aspx" target="_blank" rel="nofollow">Namecheap FreeDNS service</a>. This will set up your domain for a smooth transfer to Namecheap, for uninterrupted web presence during and after the transfer. </li> <li>Enter your domain name, for example <code>artforhealingenvironments.com</code>, and click on <b>Get DNS</b></li> <li>On the next page, click on <b>Add DNS Service For the Selected Domains</b></li> <li>Fill in the Namecheap <a href="https://manage.www.namecheap.com/myaccount/hosteddomainslist.aspx" target="_blank" rel="nofollow">Hosted Domains page</a>. It will look something like this when you are done, for the <tt>artforhealingenvironments.com</tt> domain. </li> </ol> <div style=""> <picture> <source srcset="/blog/images/dns.webp" type="image/webp"> <source srcset="/blog/images/dns.png" type="image/png"> <img src="/blog/images/dns.png" title="Namecheap Hosted Domains page" class=" liImg2 rounded shadow" style="width: 100%" alt="Namecheap Hosted Domains page" /> </picture> </div> <ol> <ol> <li>In the first IP ADDRESS/URL, put the fully resolved HTTP URL, with a trailing slash; set the record type to <b>URL Redirect</b>: <code>https://www.artforhealingenvironments.com/</code> </li> <li>In the second IP ADDRESS/URL, put the AWS bucket name, followed by the S3 site's domain; Namecheap will automatically add a period after, so you do not have to. Set the record type to <b>CNAME</b>:<code>www.artforhealingenvironments.com.s3-website-us-east-1.amazonaws.com</code> </li> <li>Namecheap's minimum TTL is 60 minutes.</li> <li>Namecheap's website automatically adds a period after each domain name.</li> </ol> </ol> <ol> <li>The MX records for email are shown at the bottom; they won't appear until you press the Save Changes button once (not shown). The MAILSERVER HOST NAME values are <code>mx1.emailsrvr.com</code> and <code>mx2.emailsrvr.com</code>. </li> <li>After the transfer completes, the FreeDNS entry will automatically be removed, and you will manage the domain on the Namecheap <a href="https://manage.www.namecheap.com/myaccount/domain-list.asp" target="_blank" rel="nofollow">Manage Domains</a> page. </li> </ol> Debugging JVM Programs on Heroku 2012-09-28T00:00:00-04:00 https://mslinn.github.io/blog/2012/09/28/debugging-jvm-programs-on-heroku <h2 id="java">Debugging Java Applications</h2> <p> Sometimes there is no substitute for debugging a remote application. The Java virtual machine provides the <a href="https://docs.oracle.com/javase/6/docs/technotes/guides/jpda/index.html" target="_blank" rel="nofollow">JPDA</a> facility for this. JPDA is flexible, and can be configured in a variety of ways. Two attachment mechanisms are supported for debugging remote applications: inbound connections, whereby a debugging process on your machine <i>attaches</i> to a remote process via designated port at a specific IP address, and outbound connections, whereby a debugging process on your local machine <i>listens</i> to a designated port for a remote process to attach to it. JPDA can be used by all JVM-based languages, such as Java, Scala, Groovy and Clojure. </p> <p> Heroku only allows one incoming port, so because an incoming port is used by Heroku to connect to the hosted app, debug connections originating from your IDE will not succeed. Instead, you must set up your Heroku application to initiate an outbound connection for debugging. This cannot be done if your app uses more than one dyno, so you must scale your Heroku back to one dyno before you can remotely debug it, like this: </p> <pre data-lt-active='false'><span class="unselectable">$ </span>heroku scale web=1</pre> <p> A Heroku app can initiate an outbound connection for debugging with or without a proxy server. The examples below use the Java debugger (<tt>jdb</tt>) and IntelliJ IDEA, but you could equally well set up a debug configuration for Eclipse in a similar manner. The settings below were used with a Play 2 application with Java and Scala controller classes. </p> <p> <b>Tip</b>: <tt>heroku restart</tt> will restart your Heroku app using your existing slug. This accomplishes the same thing as: </p> <pre data-lt-active='false'><span class="unselectable">$ </span>heroku scale web=0 <span class="unselectable">$ </span>heroku scale web=1</pre> <p> The other way you can restart your app is by building a new slug, which then automatically starts. You can do this by checking in a bogus file: </p> <pre data-lt-active='false'><span class="unselectable">$ </span>date &gt; ignoreme.txt; git add ignoreme.txt; git push heroku</pre> <p> <b>Warning</b>: Your Heroku app will crash if there is no listening process. Use the <tt>heroku logs</tt> command to check for a crash. </p> <pre data-lt-active='false'><span class="unselectable">$ </span>heroku logs 2012-09-29T18:20:53+00:00 heroku[web.1]: Starting process with command `target/start -Dhttp.port=${PORT} ${JAVA_OPTS}` 2012-09-29T18:20:55+00:00 app[web.1]: ERROR: transport error 202: connect failed: Connection refused 2012-09-29T18:20:55+00:00 app[web.1]: ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) 2012-09-29T18:20:55+00:00 app[web.1]: JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [../../../src/share/back/debugInit.c:741] 2012-09-29T18:20:55+00:00 app[web.1]: FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197) 2012-09-29T18:20:56+00:00 heroku[web.1]: Process exited with status 134 2012-09-29T18:20:56+00:00 heroku[web.1]: State changed from starting to crashed</pre> <h2 id="remote">Remote Debugging Without a Proxy Server</h2> <p> If your local machine is accessible from the Internet at an IP address (a domain such as <code>blah.no-ip.info</code> would work equally well): </p> <pre data-lt-active='false'><span class="unselectable">$ </span>jdb -listen 9999&amp; # Do not type this line if you are using an IDE # If using an IDE, start debugging now <span class="unselectable">$ </span><span class="unselectable">$ </span># Assumes that eth0 accesses the Internet; only works if you are not behind NAT <span class="unselectable">$ </span><span class="unselectable">$ </span>IPADDR=`ifconfig eth0|grep "inet addr"|awk -F: '{print $2}'|awk '{print $1}'` <span class="unselectable">$ </span><span class="unselectable">$ </span># If you are behind NAT, you will need to use a dynamic DNS and set IPADDR to the machine name instead: <span class="unselectable">$ </span><span class="unselectable">$ </span>IPADDR=mycomputer.no-ip.info # modify to suit <span class="unselectable">$ </span><span class="unselectable">$ </span># This is a really long line. I wrapped it, but you should not: <span class="unselectable">$ </span><span class="unselectable">$ </span>heroku config:add JAVA_OPTS='-Xdebug -Xrunjdwp:transport=dt_socket,address=$IPADDR:9999 -Xms512M -Xmx1024M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256M -XX:+UseCompressedOops' <span class="unselectable">$ </span>heroku restart</pre> <p> Here is what the run configuration for IntelliJ IDEA looks like. Note that debugger mode is set to <code>listen</code>, which implies <code>server="n"</code>. You need to launch this run configuration before restarting your Heroku app. </p> <div style=""> <picture> <source srcset="/blog/images/ideaListen.webp" type="image/webp"> <source srcset="/blog/images/ideaListen.png" type="image/png"> <img src="/blog/images/ideaListen.png" title="IntelliJ IDEA listening as a client" class=" liImg2 rounded shadow" style="width: 100%" alt="IntelliJ IDEA listening as a client" /> </picture> </div> <h2 id="impl">Bash Script Implementation</h2> <p>I put the bash scripts in a gist:</p> <noscript><pre>#!/bin/bash set -e export JAVA_OPTS=&quot;&quot; export JAVA_OPTS=&quot;$JAVA_OPTS -Xmx2048m -Xss512m -Xverify:none&quot; # Java 8 no longer supports MaxPermSize: # export JAVA_OPTS=&quot;$JAVA_OPTS -XX:MaxPermSize=384m&quot; </pre></noscript><script src="https://gist.github.com/mslinn/7937391.js"> </script> <h2 id="proxy">Remote Debugging With a Proxy Server</h2> <p> If your local machine is hidden from the Internet by a proxy server at <tt>blah.domain.com</tt>, open a tunnel to it from your local machine, and listen to it: </p> <pre data-lt-active='false'><span class="unselectable">$ </span># Assumes that eth0 accesses the Internet; only works if you are not behind NAT <span class="unselectable">$ </span>IPADDR=`ifconfig eth0|grep "inet addr"|awk -F: '{print $2}'|awk '{print $1}'` <span class="unselectable">$ </span># If you are behind NAT, you will need to use a dynamic DNS and set IPADDR to the machine name instead: <span class="unselectable">$ </span>IPADDR=mycomputer.no-ip.info # modify to suit <span class="unselectable">$ </span>ssh -NR *:9999:localhost:9999 $IPADDR&amp; <span class="unselectable">$ </span>jdb -listen 9999&amp; # Do not type this line if you are using an IDE</pre> <p>If using an IDE, start debugging now.</p> <pre data-lt-active='false'><span class="unselectable">$ </span># This is a really long line. I wrapped it, but you should not: <span class="unselectable">$ </span>heroku config:add JAVA_OPTS='-Xdebug -Xrunjdwp:transport=dt_socket,address=$IPADDR:9999 -Xms512M -Xmx1024M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256M -XX:+UseCompressedOops' <span class="unselectable">$ </span>heroku restart</pre> <h2 id="disablingRemote">Disabling Remote Debugging</h2> <p> The <tt>JAVA_OPTS</tt> value set earlier will remain in effect across multiple restarts of your Heroku app until you change it. To disable remote debugging, redefine the environment variable without the <tt>-Xdebug</tt> and <tt>-Xrunjdwp</tt> options: </p> <pre data-lt-active='false'><span class="unselectable">$ </span># This is a really long line. I wrapped it, but you should not: <span class="unselectable">$ </span>heroku config:add JAVA_OPTS='-Xms512M -Xmx1024M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256M -XX:+UseCompressedOops' <span class="unselectable">$ </span>heroku restart</pre> <h2 id="saving">Saving the Run Configuration</h2> <p> If you enable the <b>Share</b> checkbox in the IntelliJ IDEA run configuration dialog box, the definition will be written to <code>.idea/runConfiguration/</code>. Normally you would not check in the contents of <code>.idea/</code> to your source code repository, but this subdirectory is an exception, and because this run configuration has no local dependencies it can be shared without modification amongst all of your developer team. </p> <pre data-lt-active='false'><span class="unselectable">$ </span><b>cat .idea/runConfigurations/Heroku_remote.xml </b> &lt;component name="ProjectRunConfigurationManager"&gt; &lt;configuration default="false" name="Heroku remote" type="Remote" factoryName="Remote"&gt; &lt;option name="USE_SOCKET_TRANSPORT" value="true" /&gt; &lt;option name="SERVER_MODE" value="true" /&gt; &lt;option name="SHMEM_ADDRESS" value="javadebug" /&gt; &lt;option name="HOST" value="localhost" /&gt; &lt;option name="PORT" value="9999" /&gt; &lt;method /&gt; &lt;/configuration&gt; &lt;/component&gt;</pre> Composable Futures with Akka 2.0 2012-08-09T00:00:00-04:00 https://mslinn.github.io/blog/2012/08/09/composable-futures-with-akka-20 <div style="text-align: right;"> <picture> <source srcset="/blog/images/futuresCover226x296.webp" type="image/webp"> <source srcset="/blog/images/futuresCover226x296.png" type="image/png"> <img src="/blog/images/futuresCover226x296.png" title="Composable Futures with Akka 2.0 cover" class="right shadow rounded" alt="Composable Futures with Akka 2.0 cover" /> </picture> </div> <p> My latest book is finally complete! <a href="https://www.slinnbooks.com/books/futures/index.html" target="_blank" rel="nofollow">Composable Futures with Akka 2.0</a> features Java and Scala code examples. The book is available in PDF format. </p> <p> Akka is hot, and Akka futures are important for creating responsive applications that can scale. This book is intended for &lsquo;the rest of us&rsquo; &mdash; Java and Scala programmers who would like to quickly learn how design and integrate applications using composable futures. <p> <p> I <a href="https://www.meetup.com/sv-jug/events/50402942/" target="_blank" rel="nofollow">presented a preview of the book</a> at Googleplex April 18, 2012 to the Silicon Valley Java User Group. </p> Proposal – Mandatory Countervailing Tip 2012-08-07T00:00:00-04:00 https://mslinn.github.io/blog/2012/08/07/proposal-mandatory-countervailing-tip <p> Inequity between countries and ethnicities exploits individuals who are the primary producers and harms the individuals and societies that contain the primary consumers; it only benefits international traders. Over 99% of the world is exploited to serve less than the top 1%. This imbalance threatens world stability, as well as national, regional and local stability. </p> <p> This proposal is designed to create jobs in every country where the trade balance is negative, and is intended to reverse the exploitation of individuals in producing countries. I propose a Mandatory Countervailing Tip (MCT). It’s like a tip that you might offer someone who provides service, and whose wages are not sufficient to justify their employment. Waiters, waitresses, bellhops and musicians are paid tips for this reason. This proposal is unlike a countervailing tax because the consuming country does not keep the surcharge; instead, it would provide money directly to individuals in producing regions. The exact mechanism(s) for disbursing funds locally is not addressed in this proposal. </p> <p> The MCT would be levied at the point of purchase, like a sales tax, and would be disbursed directly to residents at the point(s) of origin of the goods who created the product. This would serve to improve the lives of all parties involved in the transaction, except the international trader. </p> <p> The MCT would be calculated such that exploited individuals themselves in the producing nation would directly receive enough money to raise their standard of living to within 25% of the standard of living for the middle 80% of the consuming region in the producing country. The money would be collected by the same government departments that collect sales tax and would be remitted every 30 days to a new agency in the UN. The entire staff of the UN agency would be required to be replaced every 2 years, and working committees and task forces in the agency would be comprised of residents of primary producers and consumers, as well as enough individuals from neutral countries to ensure impartiality and honesty. The states collecting the sales tax would be entitled to 1% of the funds collected for administrative overhead. The UN agency would be entitled to an additional 2%. The UN would be responsible for computing the MCT for every category of product sold, for every producing / consuming country. </p> <p> The sums involved would be enormous, so the potential for fraud is also enormous. To counteract this, unprecedented transparency must be implemented; this is possible because of modern computer and communications technology. Access to the data should be made public to the world at large. No sign-in should be required to download data concerning anonymized transactions and aggregated statistics. No data about individuals in producers or consumers must be available without security clearance. </p> <p> Transactions and queries requiring authorization must be audited, and auditors must be selected from presumably hostile countries and regions. For example, given that India and China are fierce competitors, they should provide the personnel to audit each other’s transactions. Those auditors would be paid from 0.002% of the total receipts. Auditors must also be reassigned every year, and may not ever be reassigned to the same region. </p> <p> Implementation need not require world-wide agreement; only one country need to decide that it wants to go ahead, and rough estimates of economic disparity would suffice to start. The first country to implement will be the first country to enjoy the benefits of a local economic resurgence. This can be done before the UN is ready, by setting up a temporary agency. </p> <p> This proposal is intended to greatly reduce the disparity between rich and poor throughout the world. It is fair because it is based on consumption, and the degree of equalization applied is proportional to the disparity between the standards of living between producer and consumer. This should greatly decrease the suicide rate of the indentured slaves who build iPads in China, while providing an opportunity for manufacturing to resurge in the US. Think of it as a regulated fair trade for world stability, prosperity and peace. </p> <p> Colonialism is based on exploitation, but capitalism does not require it. One might argue that a certain amount of exploitation is healthy because without some measure of economic disparity, undeveloped regions would not have a cost advantage and therefore never develop. This is probably true, so the question then becomes: how much economic disparity is healthy? I don’t know the answer, but a legislated policy on whatever that figure needs to be is preferable to uncontrolled exploitation. I doubt that the figure would need to be exceed 50%, and probably should be less. Current figures are in the range of 1% to 5%. </p> <p> The MCT is an example of how local, regional, national and world government can provide for the greater good. The only parties who have reason to oppose this type of proposal are those who benefit in some way from exploitation. Think of the goodwill that the recipients will feel towards their end users, and the countries that they live in. </p> Scala Existential Types and Salat 2012-08-06T00:00:00-04:00 https://mslinn.github.io/blog/2012/08/06/scala-existential-types <p> For Scala programmers, the term 'existential types' does not refer to <a href="https://en.wikipedia.org/wiki/Existentialism" target="_blank" rel="nofollow">philosophers</a>, or even authentic people. Ironically, most of the discussions of Scala's existential types that I found are too abstract to be useful to me. I learn by doing; rarely do I learn from reading an abstract treatise. I guess this means that I could be labelled an existentialist. </p> <p> Section 31.3 of <a href="https://www.artima.com/shop/programming_in_scala_2ed" target="_blank" rel="nofollow">Programming in Scala</a> has some good information on Scala's existential types. Existential types is an abstraction of Java types, and abstraction is the antithesis of existentialism... aaaanyway, here is a sentence I stole from the book:</p> <p class="rounded shadow liImg" style="padding: 1em;"><tt>Iterator[_]</tt> means the same thing as <tt>Iterator[T] forSome { type T }</tt></p> <p> <tt>forSome</tt> is the keyword which tells the compiler that an existential type is being defined. This becomes useful if upper and/or lower bounds are used to define the type. For example, you can use a lower bound to specify that the type must be a subclass of <tt>PubSubAction</tt> with the following existential type: </p> <pre data-lt-active='false'>T forSome { type T &lt;: PubSubAction }</pre> <p>That is a lot of characters, so let's give this type a name:</p> <pre data-lt-active='false'>type PubSubActionSubclass = T forSome { type T &lt;: PubSubAction }</pre> <p>We can also define a type to help us with <tt>SalatDAO</tt>:</p> <pre data-lt-active='false'>type SalatObject = T forSome { type T &lt;: AnyRef }</pre> <p>Now lets use <tt>SalatObject</tt> as the parametric type for an invocation of <tt>Manifest.classType()</tt>:</p> <pre data-lt-active='false'>val mot = Manifest.classType[SalatObject](msg.getClass)</pre> <p> This is useful because the manifest can be passed to a <a href="https://github.com/novus/salat/wiki/SalatDAO" target="_blank" rel="nofollow"><tt>SalatDAO</tt></a> constructor, which will create a DAO object for whatever type is supplied. In the following code, <tt>PubSubActionClass</tt> is just used to guarantee that the <tt>msg</tt> parameter is of the correct type; <tt>SalatDAO</tt>'s constructor is defined to accept all subclasses of <tt>AnyRef</tt>. </p> <pre data-lt-active='false'>import com.novus.salat.global.ctx abstract class PubSubAction object PubSubAction { def makeDAO[PubSubActionSubclass](msg: PubSubActionSubclass) (implicit coll: MongoCollection) = { val mot = Manifest.classType[SalatObject](msg.getClass) val mid = Manifest.classType[Int](classOf[Int]) new SalatDAO(coll)(mot, mid, ctx){} } }</pre> <p> Now you know that you do not have to hard-code the creation of a DAO for every <tt>PubSubAction</tt> subclass. You can examine the <a href="https://github.com/novus/salat/blob/master/salat-core/src/main/scala/com/novus/salat/dao/SalatDAO.scala" target="_blank" rel="nofollow">source code for <tt>SalatDAO</tt></a> if you would like to learn more. </p> <p> Assuming that <tt>psaMsg</tt> is an instance of a <tt>PubSubActionSubclass</tt>, you could call <tt>insert()</tt> as follows to do a generic insert. Because Salat has multiple methods called <tt>insert()</tt>, there is not enough type information for Scala to disambiguate the method reference unless the returned value from the single insert is stored in a variable, and that variable's type is provided: </p> <pre data-lt-active='false'>val ignoredIndex: Option[Int] = makeDAO.insert(psaMsg)</pre> <p> There is only one variant of <tt>insert()</tt> which accepts a collection, so the reference is not ambiguous and the return value can be ignored: </p> <pre data-lt-active='false'>dao.insert(Seq(psaMsg, psaMsg2, psaMsg3), WriteConcern.Normal)</pre> Pushing Notifications to Nagios from Java and Scala 2012-08-04T00:00:00-04:00 https://mslinn.github.io/blog/2012/08/04/pushing-notifications-to-nagios-from <p> I was investigating how to push notifications from JVM-based programs when I found the <a href="https://sourceforge.net/projects/nagiosappender" target="_blank" rel="nofollow">NagiosAppender</a> project. Push notifications are termed &lsquo;passive checks&rsquo; because Nagios does not poll for results. For the curious, see the Nagios Plugins &mdash; Passive Service Check section of <a href="https://www.novell.com/communities/node/4131/application-monitoring-made-easy-java-applications-using-nagios" target="_blank" rel="nofollow">Application Monitoring Made Easy for Java Applications Using Nagios</a>. </p> <p> NagiosAppender integrates Log4j or Logback with Nagios&rsquo; optional NSCA server. The only &lsquo;programming&rsquo; required is setting up configuration files for Nagios client and Nagios server, adding a new dependency, and writing appropriate log messages for forwarding to Nagios by the plugin. Unfortunately, NagiosAppender is not compatible with Akka because <a href="https://logback.qos.ch/manual/mdc.html" target="_blank" rel="nofollow">it uses MDC</a>, which uses <a href="https://docs.oracle.com/javase/7/docs/api/java/lang/ThreadLocal.html" target="_blank" rel="nofollow"><tt>ThreadLocal</tt></a> variables, which should not be used with Akka. I took the NagiosAppender project, slimmed it down, removed the Log4j interface and the MDC code, and created the <a href="https://github.com/mslinn/PushToNagios" target="_blank" rel="nofollow">PushToNagios</a> project. </p> <p> NB: The document mentions MDC without defining it. From the <a href="https://logging.apache.org/log4j/1.2/" target="_blank" rel="nofollow">Apache log4j</a> docs: &ldquo;A Mapped Diagnostic Context, or MDC, is an instrument for distinguishing interleaved log output from different sources. Log output is typically interleaved when a server handles multiple clients near-simultaneously. The MDC is managed on a per-thread basis. A child thread automatically inherits a copy of the mapped diagnostic context of its parent.&rdquo; The Logback documentation has a <a href="https://logback.qos.ch/manual/mdc.html" target="_blank" rel="nofollow">whole chapter on MDC</a>. </p> <h3 id="nsca">NSCA</h3> <p> NSCA is a Nagios add-on that allows you to send <a href="http://nagios.sourceforge.net/docs/nagioscore/3/en/passivechecks.html" target="_blank" rel="nofollow">passive check</a> results from remote hosts to the Nagios daemon running on the monitoring server. This is very useful in <a href="http://nagios.sourceforge.net/docs/nagioscore/3/en/distributed.html" target="_blank" rel="nofollow">distributed</a> and <a href="http://nagios.sourceforge.net/docs/nagioscore/3/en/redundancy.html" target="_blank" rel="nofollow">redundant/failover</a> monitoring setups. The NSCA addon can be found on <a href="http://exchange.nagios.org/directory/Addons/Passive-Checks/NSCA--2D-Nagios-Service-Check-Acceptor/details" target="_blank" rel="nofollow">Nagios Exchange</a>. For more information, see <a href="http://exchange.nagios.org/directory/Addons/Passive-Checks/NSCA--2D-Nagios-Service-Check-Acceptor/details" target="_blank" rel="nofollow">Addon &mdash; Nagios Passive Checks with NSCA</a>. </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/nsca.webp" type="image/webp"> <source srcset="/blog/images/nsca.png" type="image/png"> <img src="/blog/images/nsca.png" title="Nagios addon: Passive Checks with NSCA" class="center liImg2 rounded shadow" alt="Nagios addon: Passive Checks with NSCA" /> </picture> </div> <h3 id="installation">Installation</h3> <p>For Ubuntu:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>sudo apt-get install nagios3 nsca</pre> <p> Installs Nagios Core 3.2.3, which is outdated but compatible, and <tt>nsca 2.7.2+nmu2</tt>, which is current. The current version of Nagios Core is 3.4.1, released on 2012-05-14. Nagios starts automatically after installation, but NSCA needs to be started manually (don't do that yet, keep reading). Navigate your web browser to <a href="http://localhost/nagios3/" target="_blank" rel="nofollow">http://localhost/nagios3</a> and specify userid <tt>nagiosadmin</tt>. </p> <p>FYI, <tt>/etc/init.d/nagios3</tt> contains:</p> <pre data-lt-active='false'>DAEMON=/usr/sbin/nagios3 NAGIOSCFG="/etc/nagios3/nagios.cfg" CGICFG="/etc/nagios3/cgi.cfg"</pre> <p><tt>/etc/nagios3/nagios.cfg</tt> contains:</p> <pre data-lt-active='false'>log_file=/var/log/nagios3/nagios.log cfg_file=/etc/nagios3/commands.cfg cfg_dir=/etc/nagios-plugins/config</pre> <p><tt>/etc/nagios3/resource.cfg</tt> contains:</p> <pre data-lt-active='false'># Sets $USER1$ to be the path to the plugins $USER1$=/usr/lib/nagios/plugins</pre> <p><tt>/etc/init.d/nsca</tt> contains:</p> <pre data-lt-active='false'>DAEMON=/usr/sbin/nsca CONF=/etc/nsca.cfg OPTS="--daemon -c $CONF" PIDFILE="/var/run/nsca.pid"</pre> <p> We saw above that plugins are in <tt>/usr/lib/nagios/plugins/</tt>. I added one called <tt>check_domain_bus</tt> with permissions set to 755, and owned by <tt>nagios:nagios</tt>: </p> <pre data-lt-active='false'>#!/bin/sh echo "All OK: $1" exit 0 Configuration</pre> <p>Edit <tt>/etc/nagios3/nagios.cfg</tt> and enable external commands on line 145 so the entry looks like this:</p> <pre data-lt-active='false'>check_external_commands=1</pre> <p>Edit the last line of <tt>/etc/nsca.cfg</tt> to disable encryption:</p> <pre data-lt-active='false'>decryption_method=0</pre> <p>Define a new Nagios command called <tt>check_domain_bus</tt> in <tt>/etc/nagios3/commands.cfg</tt> by adding the following anywhere in that file:</p> <pre data-lt-active='false'>define command { command_name check_domain_bus command_line $USER1$/check_domain_bus $ARG1$ }</pre> <p> Define a template for passive services, and an instance of a passive service called <tt>domainBus</tt> that responds to the <tt>check_domain_bus</tt> command by adding the following to <tt>/etc/nagios3/conf.d/services_nagios2.cfg</tt>: </p> <pre data-lt-active='false'>define service { &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; name&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; passive-service &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; use&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; generic-service &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; check_freshness&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; passive_checks_enabled&nbsp; 1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; active_checks_enabled&nbsp;&nbsp; 0 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; is_volatile&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; flap_detection_enabled&nbsp; 0 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; notification_options&nbsp;&nbsp;&nbsp; w,u,c,s &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; freshness_threshold&nbsp;&nbsp;&nbsp;&nbsp; 57600&nbsp;&nbsp;&nbsp;&nbsp; ;12hr } define service { &nbsp;&nbsp;&nbsp; use&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; passive-service&nbsp; &nbsp;&nbsp;&nbsp; host_name&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; localhost &nbsp;&nbsp;&nbsp; service_description&nbsp;&nbsp;&nbsp;&nbsp; domainBus &nbsp;&nbsp;&nbsp; check_command&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; check_domain_bus!0 }</pre> <h3 id="usage">Usage</h3> <p>Start NSCA server, then restart Nagios:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>sudo service nsca start <span class="unselectable">$ </span>sudo service nagios3 restart</pre> <p> The custom service, called <tt>domainBus</tt>, should be viewable as a Nagios service, shown in the red rectangle below: </p> <div style=""> <picture> <source srcset="/blog/images/nagiosOK.webp" type="image/webp"> <source srcset="/blog/images/nagiosOK.png" type="image/png"> <img src="/blog/images/nagiosOK.png" title="Nagios domainBus service" class=" liImg2 rounded shadow" style="width: 100%" alt="Nagios domainBus service" /> </picture> </div> <p> Nagios will need to be restarted each time a service definition is modified. New services are shown as <tt>PENDING</tt> until they receive their first result. Passive services have no scheduled updates. </p> <h3 id="testing">Testing with the PushToNagios Java Client</h3> <p>See the <a href="https://github.com/mslinn/PushToNagios" target="_blank" rel="nofollow">PushToNagios</a> documentation.</p> <h3 id="test2">Testing With the Compiled C NSCA Client</h3> <p> Let&rsquo;s send a message and have the result displayed on the web interface. <code>send_nsca</code> is a <a href="https://sourceforge.net/projects/nagios/files/nsca-2.x/nsca-2.7.2/nsca-2.7.2.tar.gz/download" target="_blank" rel="nofollow">compiled C nsca client</a> that can be used to send a test message. Unpack <tt>nsca-2.7.2.tar.gz</tt> into a directory, and compile it: </p> <pre data-lt-active='false'><span class="unselectable">$ </span>./configure <span class="unselectable">$ </span>make install</pre> <p>Again, edit the last line of <tt>sample-config/send_nsca.cfg</tt> and change it to read:</p> <pre data-lt-active='false'>decryption_method=0</pre> <p>Create a test message in the root of the unpacked NSCA project. The format for a service check packet using NSCA contains tab characters and ends in a newline, like this:</p> <pre data-lt-active='false'>&lt;hostname&gt;[tab]&lt;svc_description&gt;[tab]&lt;return_code&gt;[tab]&lt;plugin_output&gt;</pre> <p>I am unsure if <tt>&lt;hostname&gt;</tt> refers to the Nagios host or the sending host. The allowable values for <tt>&lt;return_code&gt;</tt> are:</p> <blockquote>0 - OK state<br> 1 - Warning state<br> 2 - Error state<br> 3 - Unknown state</blockquote> <p><tt>&lt;plugin_output&gt;</tt> can be up to 512 bytes long.</p> <p>Create a text message called <tt>testCritical</tt>, with embedded tabs, that NSCA uses as a field delimiter.</p> <pre data-lt-active='false'>localhost&nbsp;&nbsp; domainBus&nbsp;&nbsp;&nbsp; 2&nbsp;&nbsp; This is a Test Error</pre> <p>Watch the Nagios log and syslog in one console:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>tail -f /var/log/nagios3/nagios.log /var/log/syslog</pre> <p>Send the test message like this in another console; let's call this the command console:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>src/send_nsca localhost -c sample-config/send_nsca.cfg &lt; testCritical</pre> <p>Notice the log output in the console with the log output:</p> <pre data-lt-active='false'>==&gt; /var/log/nagios3/nagios.log &lt;== [1343251589] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;localhost;domainBus;2;This is a Test Error ==&gt; /var/log/syslog &lt;== Jul 25 14:26:29 natty nagios3: EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;localhost;domainBus;2;This is a Test Error ==&gt; /var/log/nagios3/nagios.log &lt;== [1343251590] PASSIVE SERVICE CHECK: localhost;domainBus;2;This is a Test Error ==&gt; /var/log/syslog &lt;== Jul 25 14:26:30 natty nagios3: PASSIVE SERVICE CHECK: localhost;domainBus;2;This is a Test Error ==&gt; /var/log/nagios3/nagios.log &lt;== [1343251590] SERVICE ALERT: localhost;domainBus;CRITICAL;SOFT;1;This is a Test Error ==&gt; /var/log/syslog &lt;== Jul 25 14:26:30 natty nagios3: SERVICE ALERT: localhost;domainBus;CRITICAL;SOFT;1;This is a Test Error</pre> <p> In the web browser, click on <b>Services</b> again and notice that the status of the domainBus service is now <tt>CRITICAL</tt>, and <b>Status Information</b> now reads: </p> <pre data-lt-active='false'>This is a Test Error</pre> <div style=""> <picture> <source srcset="/blog/images/nagiosCritical.webp" type="image/webp"> <source srcset="/blog/images/nagiosCritical.png" type="image/png"> <img src="/blog/images/nagiosCritical.png" title="Status of the domainBus service is now 'CRITICAL'" class=" liImg2 rounded shadow" style="width: 100%" alt="Status of the domainBus service is now 'CRITICAL'" /> </picture> </div> <p>Create a text message called <tt>testClear</tt>, and do not forget the embedded tabs:</p> <pre data-lt-active='false'>localhost&nbsp;&nbsp; domainBus&nbsp;&nbsp;&nbsp; 0&nbsp;&nbsp; Mischief Managed</pre> <p>Send this new test message in the command console:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>src/send_nsca localhost -c sample-config/send_nsca.cfg &lt; testClear</pre> <p>The log output console should show something like this:</p> <pre data-lt-active='false'>==&gt; /var/log/nagios3/nagios.log &lt;== [1343252049] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;localhost;domainBus;0;Mischief Managed ==&gt; /var/log/syslog &lt;== Jul 25 14:34:09 natty nagios3: EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;localhost;domainBus;0;Mischief Managed ==&gt; /var/log/nagios3/nagios.log &lt;== [1343252050] PASSIVE SERVICE CHECK: localhost;domainBus;0;Mischief Managed [1343252050] SERVICE ALERT: localhost;domainBus;OK;SOFT;2;Mischief Managed ==&gt; /var/log/syslog &lt;== Jul 25 14:34:10 natty nagios3: PASSIVE SERVICE CHECK: localhost;domainBus;0;Mischief Managed</pre> <p> In the web browser, the <b>Services</b> information should automatically update after a pause of up to 90 seconds (by default), or you can click on Services to immediately see the new status. Notice that the status of the <tt>domainBus</tt> service is now <tt>OK</tt>, and <b>Status Information</b> now reads <tt>Mischief Managed</tt>. </p> <div style=""> <picture> <source srcset="/blog/images/nagiosOK.webp" type="image/webp"> <source srcset="/blog/images/nagiosOK.png" type="image/png"> <img src="/blog/images/nagiosOK.png" title="Status Information now reads 'Mischief Managed'" class=" liImg2 rounded shadow" alt="Status Information now reads 'Mischief Managed'" /> </picture> </div> <p> The third possible message status is <tt>warning</tt>. Create a text message called <tt>testWarning</tt> and do not forget the embedded tabs: </p> <pre data-lt-active='false'>localhost&nbsp;&nbsp; domainBus&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp; Do you know where your chocolate is?</pre> <p>Send the test message like this in the command console:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>src/send_nsca localhost -c sample-config/send_nsca.cfg &lt; testWarning</pre> <p> In the web browser, click on <b>Services</b> to immediately see the new status. Notice that the status of the <tt>domainBus</tt> service is now <tt>OK</tt>, and <b>Status Information</b> now reads <tt>Do you know where your chocolate is?</tt> </p> <div style=""> <picture> <source srcset="/blog/images/nagiosWarning.webp" type="image/webp"> <source srcset="/blog/images/nagiosWarning.png" type="image/png"> <img src="/blog/images/nagiosWarning.png" title="Status Information now reads 'Do you know where your chocolate is?''" class=" liImg2 rounded shadow" style="width: 100%" alt="Status Information now reads 'Do you know where your chocolate is?''" /> </picture> </div> Scala Type Parameters, Implicit Manifests and Salat 2012-08-02T00:00:00-04:00 https://mslinn.github.io/blog/2012/08/02/scala-type-parameters-implicit <p>Type parameters provide types for constructing any arguments that are implicit manifests.</p> <p>This helps when working with <a href="https://github.com/novus/salat/" target="_blank" rel="nofollow">Salat</a>. In the following code, <tt>dbo</tt> is a <tt>com.mongodb.DBObject</tt>, perhaps retrieved as the result of a <tt>find()</tt>. Salat uses <tt>_typeHint</tt> to store the fully qualified name of the persisted class; this property is returned as one of the properties of <tt>dbo</tt>. The call to Salat's <tt>grater()</tt> method converts <tt>dbo</tt> into the desired type. The type is constrained using a lower bound to be a subclass of <tt>PubSubAction</tt>.</p> <pre data-lt-active='false'>def findWithSeqNum[T &lt;: PubSubAction](seqNum: Long) (implicit manifest: Manifest[T]): Option[T] = { val klass = manifest.erasure.asInstanceOf[Class[T]] val item = myCollection.findOne( MongoDBObject("seqNum" -&gt; seqNum, "_typeHint" -&gt; klass.getName)) item match { case Some(dbo: DBObject) =&gt; Some(grater(ctx, Manifest.classType(klass)).asObject(dbo)) case None =&gt; None } }</pre> <p>When called as follows, the type parameter need not be provided because there is an explicit manifest argument:</p> <pre data-lt-active='false'>findWithSeqNum(11)(Manifest.classType(classOf("com.micronautics.Blah"))</pre> <p>When called as follows, the type parameter provides the value for constructing the implicit manifest argument:</p> <pre data-lt-active='false'>findWithSeqNum[Blah](11)</pre> <p>Of course, you can also explicitly define an implicit manifest at any scope that you desire:</p> <pre data-lt-active='false'>implicit val manifest: ClassManifest[Blah] findWithSeqNum(11)</pre> <p>If the type of the manifest is dynamic (only known at runtime), you can define manifest this way:</p> <pre data-lt-active='false'>val mtype = "com.micronautics.Blah" implicit val manifest = Manifest.classType(Class.forName(mtype)) findWithSeqNum(11)</pre> Pigs Can Fly 2011-09-30T00:00:00-04:00 https://mslinn.github.io/blog/2011/09/30/pigs-can-fly <div style="text-align: center;"> <picture> <source srcset="/assets/images/flyingPig.webp" type="image/webp"> <source srcset="/assets/images/flyingPig.png" type="image/png"> <img src="/assets/images/flyingPig.png" title="Flying pig" class="center liImg2 rounded shadow" alt="Flying pig" /> </picture> </div> <p> Details matter. If all you read are headlines and subtitles then it is likely that you are horribly misinformed. Let’s imagine that the text of an article was something like this: </p> <div class="informalNotice shadow rounded liImg"> <h2>Pigs Can Fly</h2> <p> Outfitted with 3D goggles and haptic feedback mechanisms customized for a pig&rsquo;s extremities, porcine subjects showed that they quickly adapted to visual stimulus that suggested to them that their movements caused them to fly. With training, pigs demonstrated that they could control the flight of simulated cargo planes, stunt triplanes, jet fighters and paper planes. </p> </div> <p> So, does the passage say that pigs can actually fly unassisted? Does it say that pigs <i>might think</i> they can fly? Does it discuss the possibility that pigs might be able to be trained as pilots? You would have to read the passage carefully to be able to answer those questions. If you just skim headlines then you would have no idea what the passage was about, or even if it was just humor. </p> <p> Details matter a lot when you are working with technologists. If you are easily put off by even the slightest techno-speak, then you have no business managing technologists. Specialized vocabulary summarizes complex thoughts succinctly; learn it, don&rsquo;t ask for the baby-talk version. </p> <ul> <li>You can&rsquo;t</li> <li>Summarize complex interactions</li> <li>Into a few bullet points</li> <li>Without getting it wrong</li> </ul> <p> This is true for reading and writing. If the best that a manager can grunt via a keyboard is &ldquo;you know what I mean&rdquo; then it is probable that the recipient of the email has at best only a vague idea of what the lazy, inarticulate writer meant. In truth, the writer has probably not thought through their ideas properly, and they are not clear what they meant either. </p> <p>Pick your information sources and sinks with discretion, and strive to communicate in complete thoughts.</p> <p> Here is some <a href="https://www.newscientist.com/article/dn24541-monkey-controls-two-virtual-arms-with-thoughts-alone/" target="_blank" rel="nofollow">related science</a>. </p> AMF over HTTP in another Multiverse 2011-09-21T00:00:00-04:00 https://mslinn.github.io/blog/2011/09/21/amf-over-http-in-another-multiverse <p> This silly story is taken from my book <a href="https://www.slinnbooks.com/books/serverSide/index.html" target="_blank" rel="nofollow">Flex Data Services, Hibernate and Eclipse</a>. No other part of that book is deliberately silly. I thought it would be fun to post this here. In the story, produce and food are allegories for data and a cellar is an allegory for a database. As for Klay and Adobe, well, you get the idea. </p> <p> Once upon a time, in an alternate multiverse, there might or might not have been a world called Klay. Klay was a world very like Earth, except that the peaceful inhabitants were obsessed with food and food technology. This world had perfected the art and science of growing and cooking food. Each region had its specialty, however because transportation was not mechanized. Only seasoned travelers had ever experienced the wondrous tastes and textures of the exquisite culinary creations in far-off lands. Because growing conditions varied widely, food could only be made from local produce, and thus could not be replicated outside where the food was grown. Instead, the pudgy townsfolk would gather around wandering minstrels as they sang of the nutritional marvels that they had experienced in far-off lands. </p> <p> Food was grown, prepared, cooked and eaten nearby the same fields that it was grown. The Klaylings valued freshness and texture very highly, but they found that some produce actually improved when it was aged. This produce was stored in cellars next to the fields where the produce was grown. When a chef prepared a meal, he/she would gather the ingredients and cook it to order. </p> <p> A group of rich merchants in a prosperous region realized that there was profit to be made by selling prepared food to neighboring regions. Their first attempt was to have a chef prepare a dish that resembled a souffl&eacute;, and have it delivered by horseback. The rider, holding the dish in front of him, ended up with egg all over himself. Clearly, a means of packaging the food was required. </p> <p> The merchants also had another problem. Each country had different customs regarding the shape of the containers from which they ate. It was inconceivable that a person accustomed to eating from a square bowl might eat instead from a round or oval bowl. This complication was considered serious because as any chef will tell you, presentation is very important. The food had to be prepared for the specific containers that it was served in, and once prepared could not be transferred to another container without ruining the presentation, thereby severely reducing the selling price. </p> <p> The merchants called upon the local wise woman, who meditated and went into a trance. An acolyte wrote down the muttered sentences that she uttered while communing with the Fat God. Without knowing what the words meant, she wrote down the words and presented them to the merchants after the wise woman fell asleep. The words were: </p> <p>Awesome Mouthwatering Food over Highspeed Trusty Transport Provider.</p> <p> Another acolyte, skilled in numerology, tried turning the words into sacred acronyms: AMF over HTTP. Try as he might, he could not understand any significance in those words. </p> <p>When the wise woman awoke, she drew this diagram:</p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/allegory.webp" type="image/webp"> <source srcset="/blog/images/allegory.png" type="image/png"> <img src="/blog/images/allegory.png" title="Wise Woman’s Food Delivery Service diagram" class="center halfsize liImg2 rounded shadow" style="padding: 1em" alt="Wise Woman’s Food Delivery Service diagram" /> </picture> </div> <p> The merchants gathered around the wise woman as she explained the diagram. &ldquo;The food from the fields and the cellars are brought into the kitchen as required. The meal is cooked to order, then deconstituted and packaged. A magic carpet uses AMF over HTTP to deliver the deconstituted food to the client’s kitchen, where it is reconstituted by one of the local chefs. The food is served into the appropriate containers for the patrons, and the carpet returns with the chef and payment. </p> <p> &ldquo;The key is AMF over HTTP. This magical process, yet to be invented, would consist of a tesseract, larger on the inside than it is on the outside. Within the tesseract, AMF would absorb the raw essence of the prepared food. The magic carpet would transport the tesseract almost instantaneously to the far-off land where the client lives. The magical AMF process would then reconstitute the food into the client’s desired containers.&rdquo; </p> <p> The merchants were incredulous. &ldquo;But we have never heard of AMF over HTTP. Where can we find this magical process, and the magic carpet?&rdquo; </p> <p> The old woman replied that she did not know. &ldquo;It was only a dream,&rdquo; she replied despondently. </p> <p> Fortunately, in this multiverse, AMF over HTTP does exist, and it transports data between client and servers just as the wise old woman described &mdash; and a lot more. Under the covers, AMF is used internally by Flex for many purposes. The next few chapters discuss how that can be done, and more. The Internet is our magic carpet. </p> Working With Bug Fix and Feature Branches in Git 2011-08-18T00:00:00-04:00 https://mslinn.github.io/blog/2011/08/18/working-with-bug-fix-and-feature <p> The project I am currently leading is following <a href="https://nvie.com/posts/a-successful-git-branching-model/" target="_blank" title="A Successful git Branching Model" rel="nofollow">A Successful git Branching Model</a> with only a few modifications. </p> <h3 id="create">Create a Bug Fix / New Feature Branch</h3> <p> When a developer goes to work on a new task, bug or feature they should make new branch from the <code>develop</code> branch to start their work. The new branch will be named after the Jira task ID, which in the following example is <code>EDOC-1234</code>. </p> <pre data-lt-active='false'><span class="unselectable">$ </span>git checkout develop // make develop branch current <span class="unselectable">$ </span>git pull // get latest commits <span class="unselectable">$ </span>git checkout -b EDOC-1234 develop // make new EDOC-1234 branch</pre> <p> That creates a new branch for you in your local repository named <code>EDOC-1234</code> based on the <code>develop</code> branch where everyone’s latest work is checked in. From there you can work on your new feature. For example: </p> <pre data-lt-active='false'><span class="unselectable">$ </span>echo "42"&gt;&gt;theAnswer.txt <span class="unselectable">$ </span>add theAnswer.txt <span class="unselectable">$ </span>commit -m "The answer to life, the universe and everything" <span class="unselectable">$ </span>echo "43"&gt;&gt;yetMore.txt <span class="unselectable">$ </span>add yetMore.txt <span class="unselectable">$ </span>commit -m "A present for the god who has everything: one more thing beyond 42"</pre> <h3 id="develop">Update your Bug Fix / Feature Branch with the Latest From the <code>develop</code> Branch</h3> <p> If someone commits to the <code>develop</code> branch while you are working on your feature you can get their work by (re)merging the <code>develop</code> branch to your <code>EDOC-1234</code> branch: </p> <pre data-lt-active='false'><span class="unselectable">$ </span>git checkout develop // switch to the develop branch <span class="unselectable">$ </span>git pull origin develop // pull the latest changes from the develop branch at origin <span class="unselectable">$ </span>git checkout EDOC-1234 // switch back to your feature branch <span class="unselectable">$ </span>git merge develop // add all the changes from the develop branch to your feature branch</pre> <h3 id="merge">Merge your Bug Fix / Feature Branch Back to the <code>develop</code> Branch</h3> <p> When you are done with your feature, merge it back to the <code>develop</code> branch. </p> <pre data-lt-active='false'><span class="unselectable">$ </span>git checkout develop // switch the the develop branch <span class="unselectable">$ </span>git pull origin develop // pull the latest commits to the develop branch on origin <span class="unselectable">$ </span>git merge --no-ff EDOC-1234 // merge the updates from the develop branch on origin <span class="unselectable">$ </span>git push origin develop // push your merged work to origin</pre> <h3 id="delete">Deleting a Bug Fix / Feature Branch</h3> <p> This will be done after the branch has had a code review and Q/A is complete. </p> <pre data-lt-active='false'><span class="unselectable">$ </span>git branch -d EDOC-1234 // remove your feature branch</pre> Tracking Down a Mismatched JAR 2011-08-04T00:00:00-04:00 https://mslinn.github.io/blog/2011/08/04/tracking-down-mismatched-jar <p> For debugging mismatched jars, first build a list of the jars in question, then run <a href="https://github.com/rh1247/jwhich/blob/master/releases/jwhich-1.01.zip" target="_blank"><code>jwhich</code></a> to discover the jar that provides a specific Java class. Put the jar and the script provided in the download in a directory on your <code>PATH</code> such as <tt>/usr/local/bin</tt>. </p> <p>For *nix and Mac, build the list of jars under the current directory like this:</p> <pre data-lt-active='false'>export CLASSPATH=$(find . -name \*.jar -printf %h/%p:)</pre> <p>For Cygwin, build the list of jars under the current directory like this:</p> <pre data-lt-active='false'>export CLASSPATH=$(find . -name \*.jar -printf '%h/%p;')</pre> <p>Run the script like this:</p> <pre data-lt-active='false'><span class="unselectable">$ </span>jwhich classNameToFindHere</pre> Debugging Spring's SEVERE Error ListenerStart 2011-08-04T00:00:00-04:00 https://mslinn.github.io/blog/2011/08/04/debugging-severe-error-listenerstart <p> This Spring startup error can be mystifying if you don&rdquo;t know how to tackle it. <code data-lt-active='false'>listenerStart()</code> configures and invokes application event listeners for a <code>Context</code>. Most Spring applications have several listeners, and they should be invoked in the following order. Each of them causes debug output, so if you do not see any, the first listener died: </p> <ol data-lt-active='false'> <li><code>org.springframework.web.context.ContextLoaderListener</code></li> <li><code>org.springframework.web.context.request.RequestContextListener</code></li> <li><code>flex.messaging.HttpFlexSession</code> (for Spring/Flex integration)</li> </ol> <p> If the error message appeared, one of the above did not initialize correctly. Put breakpoints in each of the listener classes&rsquo; <code>contextInitialized()</code> or <code>requestInitialized()</code> methods. </p> <p> To <a href="https://dev-answers.blogspot.com/2010/03/enable-debugtrace-level-logging-for.html" target="_blank" rel="nofollow">view debug log messages</a> for <code>StandardContext</code>, add: </p> <pre data-lt-active='false'>1catalina.org.apache.juli.FileHandler.bufferSize = -1</pre> <p> <code>SEVERE: Error listenerStart</code> messages can be debugged by setting a breakpoint at <code data-lt-active='false'>org.springframework.web.context.ContextLoaderListener</code>, line 47. </p> <pre data-lt-active='false'>contextLoader.initWebApplicationContext(event.getServletContext());</pre> <p> Step return once and wait for the container to load everything. If there is an error you will now be at <code data-lt-active='false'>standardcontext.listenerstart</code> and you will see the error in the variable window under the <code data-lt-active='false'>t</code> variable. </p> <p>For debug output, add this to <code>web.xml</code>:</p> <pre data-lt-active='false'>&lt;listener&gt; &lt;listener-class&gt; org.springframework.web.util.Log4jConfigListener &lt;/listener-class&gt; &lt;/listener&gt;</pre> Assessing Sample Code from Job Applicants 2011-05-24T00:00:00-04:00 https://mslinn.github.io/blog/2011/05/24/assessing-sample-code-from-job <p> Once again I am reviewing programmer resumes for a client. This client needs to hire an entire team, including Flex and Java programmers. Because programmers write software, I always request sample code from candidates. I do not ask stupid hypothetical questions or play mind games &mdash; I just want to assess the quality of their work. </p> <p> I request the code along with a resume. Code is truth. I do not interview candidates who do not provide sample code that passes inspection. The requirement for sample code is published in the job description. Surprisingly, most applicants do not provide sample code. I do not respond to those applications because clearly those people do not follow simple instructions. </p> <p> This is worth repeating: <b>If a job applicant wants to be considered as a candidate, they need to provide sample code along with their resume.</b> </p> <div style=""> <picture> <source srcset="/blog/images/samples_690x518.webp" type="image/webp"> <source srcset="/blog/images/samples_690x518.png" type="image/png"> <img src="/blog/images/samples_690x518.png" title="Samples of produce" class=" liImg2 rounded shadow" alt="Samples of produce" /> </picture> </div> <p> I assume that the programs that candidates show me are examples of their best work, or at least work that they are proud of. Most of my clients need programmers to develop software products or in-house applications, so efficiency and maintainability are important. </p> <p> I ask for sample code in at least two languages. It is up to the candidate to decide how much to show me, and what they show me. The more sample code they show me, the more sure I can be about my assessment. </p> <p>Here are some issues that I look for in submitted code:</p> <ol> <li>Is it difficult or expensive to maintain?</li> <li>Was the code written with the next programmer on this project in mind?</li> <li>Is logic used instead of data structures?</li> <li>Is the code neat, and does it follow a consistent pattern? I don’t care if particular formatting conventions are followed, I am looking for an ordered mind and a conscientious worker.</li> <li>Is the code testable? Were unit tests and/or integration tests provided?</li> <li>Does the code contain magic values instead of declared constants?</li> <li>Is the code inefficient (are there lots of conversions between string and int, for example)?</li> <li>Does the programmer use idioms specific to the language that it is written in?</li> <li>Are there appropriate comments, especially Javadoc / asdoc style comments?</li> <li>Are default actions or values present in the code? Defaults are tacitly understood by competent programmers, and providing them just adds noise. Code and documentation should be written with the assumption that readers will be technically competent.</li> <li>Is encapsulation violated because most methods are public?</li> <li>Are there syntax errors?</li> </ol> <p>For Adobe Flex programmers I also look for:</p> <ol> <li>Is there proper usage of the Flex component life cycle?</li> <li>Is there a heavy dependence on binding?</li> <li>Are events used properly?</li> <li>Are heavyweight Halo containers used, when Spark Groups would suffice?</li> <li>Is all logic in a controller, or is it shoved into views (or worse, item renderers)?</li> <li>Are style sheets used, or is style information explicitly coded?</li> </ol> <p> I also provide a reading and comprehension test, and ask them to critique the code I provide. Some code I provide is good, some bad. </p> <p> Many enticing resumes are simply not supported by good sample code. Would you hire a chef without tasting a sample of their food first? </p> <div style="text-align: center;"> <picture> <source srcset="/assets/images/chef.webp" type="image/webp"> <source srcset="/assets/images/chef.png" type="image/png"> <img src="/assets/images/chef.png" title="Happy chef offering a sample of her food" class="center halfsize liImg2 rounded shadow" alt="Happy chef offering a sample of her food" /> </picture> </div> Mobile and Desktop Technology Trends and Issues 2011-05-13T00:00:00-04:00 https://mslinn.github.io/blog/2011/05/13/mobile-and-desktop-technology-trends <p> As principal of Micronautics Research I develop or oversee development of desktop and mobile client/server applications. We use whatever technology is appropriate for a client project. </p> <p> I make a distinction between applications (apps) and web pages (HTML and HTML5). Over time this distinction will blur. This posting deals with current reality for apps that need to be developed and deployed using today’s technology. </p> <div style="text-align: center;"> <picture> <source srcset="/assets/images/HTML5.webp" type="image/webp"> <source srcset="/assets/images/HTML5.png" type="image/png"> <img src="/assets/images/HTML5.png" title="HTML5 logo" class="center liImg2 rounded shadow" style="height: auto; width: 300px; padding: 1em" alt="HTML5 logo" /> </picture> </div> <p> Mobile and desktop apps can be described in terms of usability, performance, development cost, and maintainability; the latter is an indicator of the time, cost and risk of modifying an existing application. Whereas desktop environments are quite stable, new mobile devices and operating system versions now appear daily. Mobile apps are therefore have a much more difficult time staying relevant than do desktop apps. Mobile app environments also vary a lot more than desktop app environments. </p> <p> There is no perfect one-size-fits-all technology for every app, and this is especially true for mobile apps. Some mobile apps must integrate tightly with the operating system; for mobile apps this currently means writing in Java for Android or Objective C for Apple iOS. The majority of mobile apps, however, must be deployed to any device that the user might happen to own; the time and cost required to target every possible device explodes unless a cross-platform technology is used. </p> <p> I’d like to share a lesson learned from migrating several desktop apps to mobile apps. To reuse the original logic and data structures from the desktop app in various mobile apps, the logic and data structures must be cleanly separated from presentation to the greatest extent possible. We have yet to see a project where rework was not required in this regard before we could begin the task of making mobile versions. </p> <p> Presentation logic for adaptable views is not much of an issue for desktop apps, but it is important when targeting devices whose resolutions and pixel densities vary greatly. Implementing the logic for adaptable views is not difficult, but designers today are generally unfamiliar with the issues of how to create <a href="https://mobile-patterns.com/" target="_blank" rel="nofollow">adaptable designs</a> for such a wide variety of platforms. We find that our interaction with designers has deepened when working on mobile projects. </p> <p> Processing power varies greatly between mobile devices. Tablets have quite a bit more power than phones, and desktops have yet more power. However, mobile devices have specialized hardware not present in most desktops, such as GPS and various forms of telephony communications; many Android devices also have a built-in bar code scanner. When adapting a desktop app to run on mobile apps, we strongly suggest using built-in hardware to the greatest extent possible. Mobile apps should obtain data from hardware or remote services instead of asking the user to type, click or swipe. </p> <p> We also enhance data structures as much as possible so less processing is required; although this is generally a good practice, today’s desktops are so powerful that careless programming often goes unnoticed. When an app is ported from a desktop environment to a mobile device, the reduced CPU and memory resources cause the app to run unacceptably slowly. </p> <p> For applications that must continue to work on desktops after they are ported to mobile devices, we recommend identifying and optimizing the application core such that it is present without changes for all environments. Sometimes clients are surprised by the results of this partitioning process because their assumptions of what is core do not align with actual commonalities between target environments. </p> Mounting compressed folders / Looping back zip files 2010-02-19T00:00:00-05:00 https://mslinn.github.io/blog/2010/02/19/mounting-compressed-folders-looping <p> Big file systems make computers run more slowly. This problem is most noticeable for laptops. On a project that I am currently working on, I am using multiple versions of three frameworks: The Flex SDK (source and ASDoc), the MicroStrategy Flex SDK and the Blaze DS source tree. The result is thousands of files that takes hours to copy and slows down disk access even when I’m not programming. </p> <h2 id="readonly">Read-only Files</h2> <p> Because these files are only read and never written, the solution is to not unzip them. Instead, the files should only be accessed from within the zip files that contains them. This capability is called a &ldquo;local loopback filesystem&rdquo; on Linux, and &ldquo;accessing compressed folders&rdquo; under Windows. </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/loopback450x338.webp" type="image/webp"> <source srcset="/blog/images/loopback450x338.png" type="image/png"> <img src="/blog/images/loopback450x338.png" title="Hardware loop back (Ethernet)" class="center halfsize liImg2 rounded shadow" alt="Hardware loop back (Ethernet)" /> </picture> </div> <p> Results are dramatic. Usage is simple. What’s not to love? </p> <h2 id="windows">Windows</h2> <p> On Windows, I use the free <a href="https://www.pismotechnic.com/pfm/ap/" target="_blank" rel="nofollow">Pismo File Mount Audit Package</a>. It provides the ability for zip files to be mounted as virtual drives. </p> <h2 id="linux">Linux</h2> <p> On Linux, use <a href="https://github.com/refi64/fuse-zip" class="code" target="_blank" rel="nofollow">fuse-zip</a>. It provides the ability for zip files to be mounted on any directory mount point. </p> Shutting down Zamples.com 2009-11-23T00:00:00-05:00 https://mslinn.github.io/blog/2009/11/23/shutting-down-zamplescom <p> I&rsquo;ve decided that it is long past the time that zamples.com should be shut off. The site got steady traffic, but it generated no revenue and demanded a lot of my time. Now that the server needs to be replaced, it is time to say goodbye. </p> <p> I first created the predecessor to Zamples in 1994 for The Internet Factory, manufacturers of the first programmable web server. Eleven years later, it is abundantly clear that no-one really cares about live online code examples, at least not enough to spend money for the privilege of using the facility. </p> <p> <a href="https://www.w3schools.com/charsets/tryit.asp?deci=128521" target="_blank" rel="nofollow">w3schools</a> built their own client-side only version recently. </p> <p> It’s rather sad to turn the switch off, but I won’t miss spending any more time and money on Zamples. </p> <div style="text-align: center;"> <picture> <source srcset="/assets/images/zamples.webp" type="image/webp"> <source srcset="/assets/images/zamples.png" type="image/png"> <img src="/assets/images/zamples.png" title="Zamples.com logo" class="center liImg2 rounded shadow" alt="Zamples.com logo" /> </picture> </div> 2009 Open Source Community Leadership Summit 2009-07-20T00:00:00-04:00 https://mslinn.github.io/blog/2009/07/20/community-leadership-summit-09 <p> The first open source <a href="https://www.communityleadershipsummit.com/" target="_blank">Open Source Community Leadership Summit</a> happened this weekend, and I was lucky enough to be able to attend. Jono Bacon, Ubuntu Community Manager, and the other volunteers did a terrific job of putting this together. I had a chance to get to know and renew acquaintances with some fascinating people from Canonical, Cisco, Collab.net, the Linux Foundation, the Linux Fund, Novell, the Open Voting Consortium, O&rsquo;Reilly Media, Red Hat, the Software Freedom Law Center, Sun and two of the the &lsquo;old men&rsquo; of the open source world, Bruce Perens and Larry Rosen. </p> <div style="text-align: right;"> <a href="https://www.rosenlaw.com/rosen.htm" target="_blank" rel="nofollow"><picture> <source srcset="/assets/images/larryRosen.webp" type="image/webp"> <source srcset="/assets/images/larryRosen.png" type="image/png"> <img src="/assets/images/larryRosen.png" title="Larry Rosen" class="right quartersize liImg2 rounded shadow" alt="Larry Rosen" /> </picture></a> </div> <div style="text-align: left;"> <a href="https://perens.com" target="_blank" rel="nofollow"><picture> <source srcset="/assets/images/brucePerens.webp" type="image/webp"> <source srcset="/assets/images/brucePerens.png" type="image/png"> <img src="/assets/images/brucePerens.png" title="Bruce Perens" class="left quartersize liImg2 rounded shadow" alt="Bruce Perens" /> </picture></a> </div> Commonwealth Club Performance 2009-06-27T00:00:00-04:00 https://mslinn.github.io/blog/2009/06/27/commonwealth-club-performance <p> Cindie Steinmetz and I performed at the San Francisco Commonwealth Club two evenings ago. We had been invited to preview a jingle I wrote entitled &ldquo;I Love Chocolate&rdquo; for the &ldquo;Women and Chocolate&rdquo; event. I played the rest of the evening, while enjoying excellent wine and chocolate. </p> <div style=""> <picture> <source srcset="/blog/images/commonwealthChocolate_5_690x518.webp" type="image/webp"> <source srcset="/blog/images/commonwealthChocolate_5_690x518.png" type="image/png"> <img src="/blog/images/commonwealthChocolate_5_690x518.png" title="Mike Slinn and Cindie Steinmetz performing at the San Francisco Commonwealth Club" class=" liImg2 rounded shadow" style="width: 100%" alt="Mike Slinn and Cindie Steinmetz performing at the San Francisco Commonwealth Club" /> </picture> </div> Presenting EmpathyWorks at a Private Equity Roundtable 2009-02-22T00:00:00-05:00 https://mslinn.github.io/blog/2009/02/22/presenting-empathyworks-at-private <p> The private equity round table vSIG is focused on applying virtual worlds to solve pressing business issues. The group meets once a month on Sandhill Road in Palo Alto, CA. Members include software entrepreneurs, VCs, academia and researchers. </p> <p class="formalNotice"> 2020 Update: the vSIG has morphed into <a href="https://www.vcr-sv.com/" target="_blank" rel="nofollow">Venture Capital Roundtable Silicon Valley</a>. </p> <p> This will be part two of two. During the last meeting I introduced the topics of modeling relationships and behavior and presented background concepts. </p> <p>The main points I made were:</p> <ul> <li>Relationships are far more than just directory entries, they are the foundation of expression and provide context for interpreting behavior</li> <li>The ability to recognize another individual is necessary in order to form a relationship with them</li> <li>Personality is perceived, not an absolute or fixed quality</li> <li>Behavior is defined by responses and is contextually dependent</li> <li>Today’s artificial beings suffer from Alzheimer’s</li> <li>It is more useful to model an entire community than a single individual</li> <li>Personas represent an individual in various contexts</li> </ul> <p> I then introduced <a href="https://www.empathyworks.ai" target="_blank" rel="nofollow">EmpathyWorks</a>, a generalized relationship and behavior modeler that I have been working on for a few years. </p> <div style="text-align: center;"> <a href="https://www.empathyworks.ai" target="_blank" rel="nofollow"><picture> <source srcset="/assets/images/robotCircle400.webp" type="image/webp"> <source srcset="/assets/images/robotCircle400.png" type="image/png"> <img src="/assets/images/robotCircle400.png" title="The EmpathyWorks Mascot" class="center halfsize liImg2 rounded shadow" alt="The EmpathyWorks Mascot" /> </picture></a> </div> Good Code is Beautiful 2008-10-15T00:00:00-04:00 https://mslinn.github.io/blog/2008/10/15/good-code-is-beautiful <p> First, I'd like to point out a few practical reasons why orderliness is important. Can you spot the bug in the following Flex code? This custom component is not as wide as expected: </p> <pre data-lt-active='false'>&lt;mx:TextArea width="100" height="100%" contextMenu="{cm}" xmlns:components="components.*" creationComplete="init()" width="100%" xmlns:mx="https://www.adobe.com/2006/mxml"&gt; ... &lt;/mx:TextArea&gt;</pre> <p> The bug occurs because two width attributes were provided, one with the value 100 pixels and one with the value 100%. Jamming a long list of attributes into an MXML tag in no particular order is an invitation to chaos. No error message is generated if an attribute is specified twice. </p> <p> Long ago I adopted the convention of ordering all attributes in alphabetical order, one per line. I would write the above like this:</p> <pre data-lt-active='false'>&lt;mx:TextArea contextMenu="{cm}" creationComplete="init()" height="100%" width="100%" xmlns:components="components.*" xmlns:mx="https://www.adobe.com/2006/mxml"&gt; ... &lt;/mx:TextArea&gt;</pre> <p> Other people write certain attributes first, like id, width and height. Whatever convention you adopt, stick to it. </p> <p> Here are the <a href="https://web.archive.org/web/20111006191913/https://opensource.adobe.com/wiki/display/flexsdk/Coding+Conventions" target="_blank" rel="nofollow">official ActionScript coding conventions</a>, as published by Adobe back in 2011, prior to canceling the Flex product. I take some of it the way the characters of &ldquo;Pirates of the Caribbean&rdquo; consider the Pirate&rsquo;s Code: &ldquo;Argh, it be more like a set of guidelines.&rdquo; </p> <div style=""> <picture> <source srcset="/blog/images/pirateCodex_690x460.webp" type="image/webp"> <source srcset="/blog/images/pirateCodex_690x460.png" type="image/png"> <img src="/blog/images/pirateCodex_690x460.png" title="The Pirate's Codex, from the Pirates of the Caribbean movie" class=" fullsize liImg2 rounded shadow" alt="The Pirate's Codex, from the Pirates of the Caribbean movie" /> </picture> </div> <h3 id="compressed">ANSI/Unix vs. Vertically Compressed Styles</h3> <p>I realize that this topic might make me flame-bait, but my experience has shown that the ANSI/Unix style of vertically spacing parenthesis increases the maintenance costs of a program.</p> <p> Functions/methods should fit on a single screen; so should data structures. Because programs written in bygone days had to be editable on a green screen, 80 columns was the widest that a program could be written. In the &lsquo;bad old days&rsquo;, it was better to let a program run to more lines in order to avoid folding code. Here is an example: </p> <pre data-lt-active='false'>private function GridContextEventHandler(event:GridContextEvent):void { if (event.menuItem=='Edit') { EditGridItem(event.selectedIndex); } else if (event.menuItem=='Remove') { RemoveGridItem(event.selectedIndex); } }</pre> <p> A vertically compressed style is much more easily read. When the Python programming language was first introduced it was initially controversial in part because it used indentation to define control structure. Python programs do not need braces for scoping as a result. Why not write ActionScript in the same manner, so parenthesis are virtually redundant? The Sun/Java convention is a good example of this formatting convention. The above program fragment would be written like this using a vertically compressed style: </p> <pre data-lt-active='false'>private function GridContextEventHandler(event:GridContextEvent):void { if (event.menuItem=='Edit') { EditGridItem(event.selectedIndex); } else if (event.menuItem=='Remove') { RemoveGridItem(event.selectedIndex); } }</pre> <h2 id="enforcing">Enforcing Style</h2> <p> Coding style conventions can be automatically applied by installing the Flex Pretty Printer plugin for Eclipse / Flash Builder. If you like, you can importing <a href="https://www.slinnbooks.com/books/FlexFormatter.properties" target="_blank" rel="nofollow">my formatting rules</a>, which enforce the vertically compressed style above. </p> <h3 id="disregard">Complete Disregard for Style</h3> <p> Recently I examined a programming candidate&rsquo;s programming style. He didn&rsquo;t get the job. Here is how he wrote the above code: </p> <pre data-lt-active='false'>private function GridContextEventHandler(event:GridContextEvent):void{ if(event.menuItem=='Edit') EditGridItem(event.selectedIndex); else if(event.menuItem=='Remove') RemoveGridItem(event.selectedIndex); }</pre> <p> The rest of his code did not work very well. Seems like from the way he wrote it that he didn't really care. No surprise that someone else got the job. </p> Breathless Delirium 2008-09-24T00:00:00-04:00 https://mslinn.github.io/blog/2008/09/24/breathless-delerium <p> In his keynote presentation to the Agile 2008 conference in Toronto entitled &ldquo;<a href="https://medium.com/@MrAlanCooper/how-far-have-we-come-792a80625c94" target="_blank" rel="nofollow">The Wisdom of Experience</a>&rdquo;, Alan Cooper had a great sentence that described the &lsquo;first to market&rsquo; goal behind some products: </p> <p> <i>&ldquo;There is no large group of people out there waiting in a breathless delirium to purchase your lousy product sooner rather than later.&rdquo;</i> </p> <p> Sure, it is terrific to be first. But the product still has to be good. </p> <p> Alan also makes some good points about design, engineering and construction. He took the long way around, because he doesn't get into his main topic, interactive design, until slide 75. Requirements are discussed starting at slide 87. In the slide entitled "Requirements are not design", Alan says: </p> <ul> <li>Giving people what they say they desire does not result in success.</li> <li>Your customers are not the same as your users.</li> <li>Neither your customers nor your users know what they want or even what they do.</li> <li>What people tell you has little bearing on the truth.</li> <li>Good user experience is not dependent on features</li> <li>Radically different products can have identical features.</li> <li>A list of features is not the same as the design of behavior.</li> <li>Expertise in a subject does not correlate to expertise in designing software behavior.</li> </ul> <p> Philosophy starts on slide 92. I like philosophy, it is one of the foundations of architecture. </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/geeklove.webp" type="image/webp"> <source srcset="/blog/images/geeklove.png" type="image/png"> <img src="/blog/images/geeklove.png" title="Geek love" class="center liImg2 rounded shadow" alt="Geek love" /> </picture> </div> Cult of the Software God 2008-04-28T00:00:00-04:00 https://mslinn.github.io/blog/2008/04/28/cult-of-software-god <p> I invented a &lsquo;religion&rsquo; in the mid-80s. Admittedly, it was done in jest, but I find that I make frequent reference to it and so it deserves a mention lest people I converse with find me just babbling. Have no fear, other whacked-out religions exist with high-profile devotees (you know some, I&rsquo;m sure) so I&rsquo;m in good company. </p> <p> The religion is called &ldquo;The Cult of the Software God&rdquo;, and this god oversees technical and financial success for software entrepreneurs and IT workers. Like many ancient religions of days gone by, the Software God requires sacrifices, which amounts to the destruction of software media. At one time he/she/it was content when I would hold a 5.25&rdquo; floppy disk over a garbage can and cut it up, while intoning praises to &ldquo;the great Software God&rdquo; and praying that the bits stored on the media be returned to their maker, but this is a God of the Geeks, and he/she/it has kept up with advances in storage media. </p> <p> When 3.5" diskettes became available, the Software God soon lost interest in clunky old floppy disks and demanded sacrifices of the smaller rigid diskettes instead. I, as High Priest of the Cult, interpreted the wishes of the Software God and used a Sacred Hammer stored on a bench near our diskette duplicating station to smash up diskettes that had write errors during the duplication process. &ldquo;Oh great Software God&rdquo;, I would say. &ldquo;We humbly beseech you to accept this offering and pray that you bring abundance and a measurable quality improvement to our next release.&rdquo; </p> <p> Time moved on, I moved to other ventures, and yet I never abandoned my role as High Priest of the Cult of the Software God. Occasionally I would snap CDs in half in the name of the Software God when a write error occurred, and moved on to DVDs. I even used a blowtorch on a RLL-encoded 5.25&rdquo; hard drive and ground my heel on a flash drive, while singing praises to the Software God. </p> <p> You might think I had too much time on my hands. Certainly I am at least eccentric. Yet there is more! </p> <p> As High Priest of the Cult of the Software God, I became aware that he/she/it is also fed by <a href="https://askubuntu.com/a/12110/58760" target="_blank"><code>/dev/null</code></a>. Where do you think bits piped to <code>/dev/null</code> go, anyway? Why, they nourish the Software God, of course! You can turn this otherwise wasted gift to the Software God into a sacrificial offering simply by making a prayer as the bits are piped. Of course, since the Software God is a god for the geeks and by the geeks, automation counts. That is why I wrote a program in the late 90s (since lost) that would print out any prayer you desired on the screen, over and over... rather like how prayer wheels work. </p> <p> Next time you really need your computer to not crash while running a semi-stable program, or have a serious virus infestation, remember the Cult of the Software God. He/she/it is virtually there for you. &#x1F609; </p> AAAI Symposium 2008-03-26T00:00:00-04:00 https://mslinn.github.io/blog/2008/03/26/aaai-symposium-for-emotion-personality <p>This week I am attending the <b>AAAI Symposium for Emotion, Personality and Social Behavior</b> at Stanford. The symposium provides a forum for an interdisciplinary discussion of modeling affect and personality in social behavior. Attendees include researchers studying social computing, virtual reality, game design, robotics, believable agents, affective computing, psychotherapy and the arts. Panel discussions discussed successful interactions between artificial beings and humans.</p> <p>Some of the discussion topics that I am interested in include:</p> <ul> <li>How can compelling artificial characters be designed?</li> <li>How to facilitate social interaction between humans, and / or artificial beings?</li> <li>What are the components in the software toolbox?</li> <li>What are the emerging standards in affective artificial characters, robots and systems?</li> <li>What architectural designs work best?</li> <li>What knowledge base designs work best?</li> </ul> <h2 id="scheutz">Believability of Robot Affect</h2> <p> Matthias Scheutz, Associate Professor of Informatics at Indiana University, presented a paper entitled &ldquo;<b>Empirical Investigations into the Believability of Robot Affect</b>&rdquo;. During the discussion afterwards he suggested that people are more critical of <a href="https://www.google.com/url?sa=t&amp;ct=res&amp;cd=2&amp;url=http%3A%2F%2Fwww.animatronics.org%2F&amp;ei=PLPrR_LFFJjgpgTfopx8&amp;usg=AFQjCNHUBENaXpnG15KYWrIuyDqpF-Eh_g&amp;sig2=kOGppoXBnh9Zuu8l3bcG4w" target="_blank" rel="nofollow">animatronic</a> (physical) entities, compared to characters that are merely imaged. Seems that the bar for believable behavior for robots is higher than for characters which are merely rendered on a computer screen because people don't expect a rendered character to be real anyway. </p> <h2 id="tapus">Socially Assistive Robots</h2> <p>Adriana Tapus&rsquo; presentation, &ldquo;<b>Socially Assistive Robots: The Link between Personality, Empathy, Physiological Signals and Task Performance</b>&rdquo; provided me two pieces of interesting common-sense information:</p> <ul> <li>A robot that challenges the user during therapy rather than offering praise will be preferred by extroverts;</li> <li>A robot that offers nurturing praise rather than challenging-based motivation by introverts.</li> </ul> <p> In the discussion that followed Adriana&rsquo;s talk, mention was made of (uncited) research which found that extroverts are more sensitive to robot personalities because they seek outside stimulus; introverts are self-stimulated. It seems logical that when a robot first encounters a person about which nothing is known regarding their personality, the robot should display mildly extroverted behavior, and then adapt according to social cues. </p> <h2 id="olsen">Emotions for Strategic Real-time Systems</h2> <p><a href="http://www.cs.loyola.edu/~olsen/" target="_blank" rel="nofollow">Megan Olsen</a> discussed how adding an emotion framework to game engines improved performance in her talk entitled &ldquo;<b>Emotions for Strategic Real-time Systems</b>&rdquo;. The emotional framework models fear and frustration in the form of emotional maps, which are overlaid on a physical map of terrain. Emotions diffuse out and are picked up by co-operating agents nearby. Emotions have an intensity value which linearly decays over time. She showed an interesting video which illustrated the aggregated emotions of many agents over time. She also showed that emotions can cause groups of synthetic individuals to be more successful in combat. Different game engines responded differently to each emotion; some responded better to the addition of the ability to model fear, which others responded better to the addition of frustration, and <a href="https://globulation2.org/wiki/AINicowar" target="_blank" rel="nofollow">NicoWar</a> was able to benefit from the addition of both emotions. </p> <h2 id="hudlicka">Emotion Modeling 101</h2> <p> <a href="https://works.bepress.com/eva_hudlicka/" target="_blank" rel="nofollow">Dr. Eva Hudlicka</a> of Psychometric Associates spoke on &ldquo;<b>Emotion Modeling 101</b>&rdquo;. Emotion modeling incorporates emotion expression, recognition, generation, and the effect on agent behavior. One could model feelings, moods, emotions, affective states and personality traits; some people mean all of the above when discussing modeling emotion. As well, attitudes and preferences can be modeled. Interpersonal roles include: social coordination, rapid communication of intent; intrapsychic roles include: motivation, homeostatis, and adaptive behavior. Emotions are manifested across multiple interacting modalities: somatic / physiological, cognitive / interpretive, behavioral / motivational and experiential / subjective. Unfortunately, the literature on emotional models is inconsistent and terms are unclear. Eva suggested we view emotion models in terms of emotion generation and effects; that emotion modeling building blocks be identified. Emotion is generated from stimuli and appraisal by the individual; Eva briefly discussed both topics and the need to standardize the process of mapping stimulus to emotion. </p> <h2 id="camurri">Coffee With Dr. Antonio Camurri</h2> <p>During a coffee break, I chatted with <a href="http://www.infomus.org/people/person.php?name=acamurri" target="_blank" rel="nofollow">Dr. Antonio Camurri</a>, Associate Professor at the University of Genoa, Italy. Seems we both share a passion for boats. Dr. Camurri mentioned EyesWeb, an open software platform project that he leads, that enables the development of real-time multimodal distributed interactive applications. </p> <h2 id="wrapup">Wrapup</h2> <p>In the wrap-up session, I drew attention to one of the issues mentioned in the preface to the Technical Report that the Symposium was to intended to address: &ldquo;What are the emerging standards in affective artificial characters, robots and systems?&rdquo; One member of the audience (Rosamaria Barone?) mentioned some standards work for characters (unfortunately, I didn't write it down), then an awkward silence followed. I believe that standards will be key to artificial personality and emotion moving from the lab to mainstream society. </p> IKVM.NET – Java applications on .NET 2008-01-16T00:00:00-05:00 https://mslinn.github.io/blog/2008/01/16/ikvmnet-java-applications-on-net <p> I am building the first implementation of <a href="https://www.empathyworks.ai" target="_blank" rel="nofollow">EmpathyWorks&trade;</a> as a Java library packaged as a JAR. After looking around for a while to understand how EmpathyWorks might run on .NET, I found <a href='https://www.ikvm.net/' target="_blank" rel="nofollow">IKVM.NET</a>. </p> <h2 id="ikvm">IKVM Works Well</h2> <p> IKVM is an open source project that converts Java JARs into a .NET assembly. I was quite skeptical, but after downloading IKVM I was able to convert EmpathyWorks into an EXE that ran on Windows XP in about ten minutes. </p> <p> IKVM also has a mechanism for converting .NET stubs into JARs, important when developing .NET code in an IDE so that Intellisense tooltips appear for code completion. I am next going to try converting the Microsoft Robotics Studio runtime to a JAR so I can try writing a sample MSR application that co-operates with EmpathyWorks. </p> <p> There is a brief note in the IKVM documentation that says debugging from Java in this circumstance does not work. Hmm, what about the other way... can one debug the .NET application from Visual Studio, and reach right into the converted Java code? I suppose no source code would be available. </p> <hr /> <h2 id="update1">Update Jan 18/08</h2> <p>I built a DLL from Java using IKVM's <code>-target:library</code> option. Visual C# 2008's object browser happily showed all the public objects and methods in the DLL. Once I discovered the &ldquo;Add to References&rdquo; button in the object browser my test C# program was able to call into the DLL. Most cool!</p> <hr /> <h2 id="update2">The End of IKVM.NET </h2> <p> Jeroen Frijters, the principal author of IKVM.NET <a href="https://weblog.ikvm.net/CommentView.aspx?guid=33ea525f-a291-418a-bd6a-abdf22d0662b" target="_blank" rel="nofollow">posted the following</a> Apr 21/17. </p> <div class="shadow rounded liImg" style="padding: 1em;"> After almost fifteen years I have decided to quit working on IKVM.NET. The decision has been a long time coming. Those of you that saw yesterday’s Twitter spat, please don’t assume that was the cause. It rather shared an underlying cause. I’ve slowly been losing faith in .NET. Looking back, I guess this process started with the release of .NET 3.5. On the Java side things don’t look much better. The Java 9 module system reminds me too much of the generics erasure debacle. <br /><br /> I hope someone will fork IKVM.NET and continue working on it. Although, I’d appreciate it if they’d pick another name. I’ve gotten so much criticism for the name over the years, that I’d like to hang on to it 😊 <br /><br /> I’d like to thank the following people for helping me make this journey or making the journey so much fun: Brian Goetz, Chris Brumme, Chris Laffra, Dawid Weiss, Erik Meijer, Jb Evain, John Rose, Mads Torgersen, Mark Reinhold, Volker Berlin, Wayne Kovsky, The GNU Classpath Community, The Mono Community. <br /><br /> And I want to especially thank my friend Miguel de Icaza for his guidance, support, inspiration and tireless efforts to promote IKVM. <br /><br /> Thank you all and goodbye. </div> I'm a Dragon on a Business Plan Panel 2007-11-21T00:00:00-05:00 https://mslinn.github.io/blog/2007/11/21/im-dragon-on-business-plan-panel <p> A few months ago the Canadian Consulate in Silicon Valley asked me if I would be interested in working with Canadian entrepreneurs. I was happy to oblige. Since then the Canadian National Research Council has been using me to provide guidance to young entrepreneurs. </p> <p> I was asked by <a href="https://www.linkedin.com/in/eanjackson/" target="_blank" rel="nofollow">Ean Jackson</a>, an instructor at <a href="https://www.sfu.ca/" target="_blank" rel="nofollow">Simon Fraser University</a> in Burnaby, BC to sit on a <a href="https://bus477.com/class-videos/" target="_blank" rel="nofollow">&ldquo;Dragon Panel&rdquo;</a> to evaluate student projects. The 4th year undergraduate business students enrolled in the New Ventures Planning Program share the course with 4th year Engineering students at SFU. </p> <p> One of the &ldquo;Suits&rdquo; is teamed up with three &ldquo;Geeks&rdquo; enrolled in the Engineering program. The Geeks come up with a technology idea and the Suits and Geeks work together to create a business plan. That business is entered into the Ken Spencer Business plan competition at SFU where cash prizes are offered. </p> <p> Today (November 21, 2007) is the final class. The 14 Suits will pitch their business ideas to a panel of &ldquo;investors&rdquo;. I am acting in the role of a prospective investor; each of the &ldquo;investors&rdquo; will grade the pitch. Sounds like fun! I'm looking forward to hearing the presentations this afternoon. </p> Digital Mentat 2007-09-23T00:00:00-04:00 https://mslinn.github.io/blog/2007/09/23/digital-mentat <div class="shadow rounded liImg"> <p class="notice">It is by caffeine alone I set my mind in motion.<p> <p class="notice">It is by the Beans of Java that thoughts acquire speed, the hands acquire shaking, the shaking becomes a warning.<p> <p class="notice">It is by caffeine alone I set my mind in motion.<p> </div> <p style="margin-left: 1em;">&mdash; Stolen from source unknown, in the SDM<br /> &mdash; After the Mentat Mantra of Dune by Brian Herbert</p> <p> It's just possible that I might have been spending too much time programming <a href='https://www.empathyWorks.ai' target="_blank" rel="nofollow">EmpathyWorks</a> lately! </p> Singularity Summit 2007-09-09T00:00:00-04:00 https://mslinn.github.io/blog/2007/09/09/singularity-summit <p> I'm at the Singularity Summit today, after spending two months on my sailboat in B.C. Quite a change in venue! </p> <p> As you can tell by reading this blog, I'm quite interested in AI. The theme of 'friendly intelligence' strikes a chord in me. I'm currently working on an artificial personality simulator (<a href='https://www.empathyworks.ai/' rel="nofollow" target="_blank">EmpathyWorks</a>), a project that I've been noodling on for over thirty years. Hopefully I'll get a chance to chat with various people here at the conference about integrating artificial personality into a whole artificial organism. </p> 2nd Annual Silicon Valley Ruby Conference 2007-04-25T00:00:00-04:00 https://mslinn.github.io/blog/2007/04/25/2nd-annual-silicon-valley-ruby <p> Whew, it is a wrap! The Second Annual Silicon Valley Ruby Conference is done, and I'm just about finished doing post-production on the videos of the talks. I'll hand a DVD to SDForum to make arrangements for attendees of the event to view the videos. </p> <p> Slides for most of the presentations are available at <a href="https://www.slideshare.net" target="_blank" rel="nofollow">slideshare.net</a>. </p> <p> 2007 seems to be shaping up as the Year of the Ruby/RoR IDE. Later this year commercial-quality IDEs from Sun and CodeGear will join Aptana's RadRails IDE. All three of these software publishers presented at this year's event. </p> <p> I ran a quick survey by a show of hands, and asked 24 questions. Here are a few results: </p> <table border="1" cellpadding="5" cellspacing="0" class="liImg"><tbody> <tr> <td style="text-align: left; padding: 5px;">Which of you use RoR to make web sites but don't care about Ruby?</td> <td align="right" style="padding: 5px;">25%</td> </tr> <tr> <td style="text-align: left; padding: 5px;">Which of you use Dreamweaver, Illustrator or Photoshop?</td> <td align="right" style="padding: 5px;">17%</td> </tr> <tr> <td style="text-align: left; padding: 5px;">Which of you use a Mac?</td> <td align="right" style="padding: 5px;">45%</td> </tr> <tr> <td style="text-align: left; padding: 5px;">Which of you use Linux?</td> <td align="right" style="padding: 5px;">35%</td> </tr> <tr> <td style="text-align: left; padding: 5px;">Which of you use Windows?</td> <td align="right" style="padding: 5px;">50%</td> </tr> </tbody></table> <p> It's been fun leading the organization of these first two years of this event, but this year will be my last. </p> Announcing Micronautics Research 2006-10-17T00:00:00-04:00 https://mslinn.github.io/blog/2006/10/17/micronautics-research-is-now-officially <div style="text-align: left;"> <a href="https://www.EmpathyWorks.ai" target="_blank" rel="nofollow"><picture> <source srcset="/blog/images/robotCircle.webp" type="image/webp"> <source srcset="/blog/images/robotCircle.png" type="image/png"> <img src="/blog/images/robotCircle.png" title=" Cool robot mascot, huh? I'm going to use it for EmpathyWorks&trade;, which at this point is still just a twinkle in my eye. " class="left liImg" alt=" Cool robot mascot, huh? I'm going to use it for EmpathyWorks&trade;, which at this point is still just a twinkle in my eye. " /> </picture></a> <figcaption class="" style="width: 100%; text-align: center;"> <a href="https://www.EmpathyWorks.ai" target="_blank" rel="nofollow"> Cool robot mascot, huh? I'm going to use it for EmpathyWorks&trade;, which at this point is still just a twinkle in my eye. </a> </figcaption> </figure> </div> A physical metaphor for IT innovation 2006-06-23T00:00:00-04:00 https://mslinn.github.io/blog/2006/06/23/a-physical-metaphor-for-it-innovation <p>I got an email today from a friend, an innovative engineer:</p > <div class='liImg rounded shadow' style="padding: 1em"> <p>Hi Mike, <p>The other day, I had a reasonably good idea for a startup. And I ran it by a friend, who said "Why would the enterprise buy that?"</p> <p>I replied "Much better than existing solutions."</p> <p>He said "Unless it's a 10x, nope. And, while it is better, it's not a 10x." </p> <p>I thought "That's gotta be wrong."</p> <p>But then it occurred to me. In spite of having worked as a consultant to enterprises, and in spite of having run professional services for an enterprise software company, I still don't have a very good model of enterprise software adoption. To wit:</p> <ul> <li>What makes a new piece of software or IT infrastructure compelling for the enterprise?</li> <li>When do enterprises decide to upgrade or change infrastructure?</li> </ul> <p>Do you know of any references or models?</p> </div> <p>Here is what I told him:</p> <p>A 10x return on investment applies to investors (although 7x works too). If the sales pitch is predicated on cost reduction (optimizing operations), then 5-10x would apply and is most interesting when the client's business is relatively mature.</p> <p>Increasing client sales (optimizing business development) needs to be presented along the client's strategic direction, which might be a moving target. 2x or 3x return would suffice if the timing between a vendor's presentation and a client's view of their strategic direction aligned. This would be difficult for the vendor to guess without incredible intelligence on specific targets.</p> <p>CIOs are risk averse, so solutions that do not require vetting new servers and client-side software are easier. When selling into a datacenter, a 10x return might not be enough if security issues are not mitigated. On the other hand, repurposing existing investments are much easier to sell: a 2x or 3x return might be sufficient.</p> <p>In other words: inertia is a dominant force of nature that also applies in IT. If you consider trends to be acceleration (positive or negative), then the net force driving the inertia is composed of external and internal factors. Those factors might be political, economic, technical, etc. Newton's <code>F=ma</code> can be restated in entrepreneurial terms as:</p> <div style="padding-left: 3em;"> net force = investment times rate of adoption</div> <p>or, more to the point:</p> <div style="padding-left: 3em;"> rate of adoption = net force / investment</div> <p>The investment (mass) a client has made in developing and deploying their intellectual property; training personnel; capital investment in infrastructure; and intimate knowledge of the current system. ("The devil you know".)</p> <p>For entrepreneurs that want to convince clients that a new technology that is worth adopting, continuing our physics metaphor may be illuminating. Let's equate profitability with speed, and technology platform with direction. Thus velocity is the cost-effectiveness of a given technical platform. It follows that in order to change a company's direction, and increase their profitability (speed), the client will need to experience a strong enough net force over a long enough period of time to overcome inertia and justify the risk and the investment. In addtion, the prospect will only understand new information in their current context, which might not use the same dictionary that an entrepreneur uses, since they tend to live in the future.</p> Success! Silicon Valley Ruby Conference 2006-04-24T00:00:00-04:00 https://mslinn.github.io/blog/2006/04/24/success-silicon-valley-ruby-conference <p> After months of planning, the big weekend arrived and a great time was had by all. Thank you, presenters! </p> <p>I organized the event and chaired the second day of the conference.</p> <p>See you next year...</p> Silicon Valley Ruby Conference 2006-02-21T00:00:00-05:00 https://mslinn.github.io/blog/2006/02/21/silicon-valley-ruby-conference <p> SDForum just announced the <a href="https://archive.upcoming.org/event/silicon-valley-ruby-conference-59161" target="_blank" rel="nofollow">Silicon Valley Ruby Conference</a>. I am one of the two co-chairmen. </p> <p> We expect the event to sell out very quickly. Some great speakers have been lined up, it's quite inexpensive and only 210 people can attend. </p> <p>See you there!</p> <p> Update: Andrew Burke wrote up some good notes for <a href="http://www.andrewburke.me/blogposts/15" target="_blank" rel="nofollow">day 1</a> and <a href="http://www.andrewburke.me/blogposts/16" target="_blank" rel="nofollow">day 2</a> of the conference. </p> Instantiating Java Inner Classes 2005-12-30T00:00:00-05:00 https://mslinn.github.io/blog/2005/12/30/instantiating-java-inner-classes <p> I've been writing Java code for years. Today I learned something new. I have been adding a new feature to Zamples that was best expressed as a doubly nested inner class. The hierarchy looks like this: </p> <pre data-lt-active='false'>public class CodeRange { public CodeRange(String str) { /* ... */ } public class RangeSpec { public RangeSpec(int i, int j) { /* ... */ } public RangeSpec(RangeItem start, RangeItem end) { /* ... */ } public class RangeItem { public RangeItem(String str, int i) { /* ... */ } } } }</pre> <p>When it came time to write JUnit tests, I needed to instantiate the hierarchy. The syntax surprised me:</p> <pre data-lt-active='false'>CodeRange.RangeSpec.RangeItem range = new CodeRange("test").new RangeSpec(1, 1).new RangeItem("middle", 3);</pre> <p>Not exactly intuitive, eh? It gets even more interesting when trying to write unit tests for the second RangeSpec constructor, the one that accepts two <code>RangeItems</code>. I found I had to create a protected no-args constructor for <code>RangeSpec</code> with an empty body, plus a method within <code>RangeSpec</code> to create <code>RangeItems</code> on demand:</p> <pre data-lt-active='false'>/** For JUnit only */ protected RangeSpec() {} /** For JUnit only */ protected RangeItem newRangeItem(String searchString, int offset) { return new RangeItem(searchString, offset); }</pre> <p>Now I could write my test case setup code:</p> <pre data-lt-active='false'>codeRange = new CodeRange(code, ""); CodeRange.RangeSpec.RangeItem rangeItemStart = codeRange.new RangeSpec().newRangeItem("middle", 3); CodeRange.RangeSpec.RangeItem rangeItemEnd = codeRange.new RangeSpec().newRangeItem("back", 0); CodeRange.RangeSpec rangeSpec = codeRange.new RangeSpec(rangeItemStart, rangeItemEnd); /* ... */</pre> <p> Everything I read about doubly nested classes amounted to a warning to the effect that they should be avoided. I never had occasion to need this type of solution before, however the particular problem I am solving yields very nicely to this approach. It is simple, elegant and efficient. Don't believe everything you read (except this blog!) &#128521;. </p> Zamplized Ruby User’s Guide 2005-12-16T00:00:00-05:00 https://mslinn.github.io/blog/2005/12/16/zamplized-ruby-users-guide <p> Santa's elves have been cheering us on at Zamples! Just in time for the holidays, Zamples v2.6 is now live. </p> <p> We've &ldquo;Zamplized&rdquo; the Ruby User&rsquo;s Guide and made it available for free. Now you can curl up with a glass of eggnog and sit by the fire with your laptop, while learning Ruby, the hottest new programming language today. </p> <p> Zamples&rsquo; live code examples let you &ldquo;learn by doing&rdquo; and give you a hands-on and foolproof experience. There is no faster, easier way to gain experience with a language or programming interface than by interacting with a Zamplized document. </p> <p> Best of the season! </p> <div style="text-align: center;"> <picture> <source srcset="/assets/images/zamples.webp" type="image/webp"> <source srcset="/assets/images/zamples.png" type="image/png"> <img src="/assets/images/zamples.png" title="Zamples logo" class="center liImg2 rounded shadow" alt="Zamples logo" /> </picture> </div> Programmatic IM Generation 2005-12-16T00:00:00-05:00 https://mslinn.github.io/blog/2005/12/16/programmatic-im-generation <p> I thought it would be fun to create a live code example using Zamples that would generate an IM (instant message). </p> <p> The following Java code example uses Jive Software&rsquo;s Smack library to send a message to a Jabber user with a Gaim client. You will need to put real values in for <tt>from</tt>, <tt>password</tt> and <tt>to</tt> before a message will be sent. BTW, if the userid and password are wrong, you&rsquo;ll get an odd error message. Since the program fragment automatically compiles and runs after you click on the <b>Try It!</b> button, the error message will appear on the right. </p> <p> Once you provide the <tt>to</tt> and <tt>from</tt> user ids and <tt>password</tt> and rerun the code fragment, a message will be sent. I tested this by sending myself a message and found that <tt>from</tt> and <tt>to</tt> were the same. </p> <table> <tbody> <tr> <td> <form action="" class="zform" method="post" target="TomcatMain"> <input name="cmd" type="hidden" value="XMPPConnection connection = new XMPPConnection(&quot;jabber.org&quot;); connection.login(&quot;from&quot;, &quot;password&quot;); connection.createChat(&quot;to@jabber.org/Gaim&quot;).sendMessage(&quot;Howdy!&quot;);"> <input name="imports" type="hidden" value="import org.jivesoftware.smack.*;"> <input name="applet" type="hidden" value=""> <input name="format" type="hidden" value="2"> <input name="pre" type="hidden" value="false"> <input name="/stylesheet" type="hidden" value="StyleSheet.css"> <input class="zform" type="submit" value="Try It!"></form> </td> <td> <pre style="text-align: left; padding: 0.5em;">XMPPConnection connection = new XMPPConnection("jabber.org"); connection.login("from", "password"); connection.createChat("to@jabber.org/Gaim").sendMessage("Howdy!");</pre> </td> </tr> </tbody> </table> <br /> <div style="text-align: center;"> <picture> <source srcset="/assets/images/zamples.webp" type="image/webp"> <source srcset="/assets/images/zamples.png" type="image/png"> <img src="/assets/images/zamples.png" title="Zamples logo" class="center liImg2 rounded shadow" alt="Zamples logo" /> </picture> </div> Zamples REST Interface 2005-12-15T00:00:00-05:00 https://mslinn.github.io/blog/2005/12/15/zamples-rest-interface <p> Wouldn&rsquo;t it be great to be able to build in a facility for live code examples into your favorite software tools? We thought so, and are proud to announce the Zamples REST Interface. Now you can use any programming language you choose to embed Zamples&rsquo; functionality as an IDE plugin or online learning environment. </p> <div style="text-align: center;"> <picture> <source srcset="/assets/images/zamples.webp" type="image/webp"> <source srcset="/assets/images/zamples.png" type="image/png"> <img src="/assets/images/zamples.png" title="Zamples logo" class="center liImg2 rounded shadow" alt="Zamples logo" /> </picture> </div> I Want to Build a Girl 2005-07-28T00:00:00-04:00 https://mslinn.github.io/blog/2005/07/28/i-want-to-build-girl <div style="text-align: center;"> <picture> <source srcset="/blog/images/cyborg-woman_450x246.webp" type="image/webp"> <source srcset="/blog/images/cyborg-woman_450x246.png" type="image/png"> <img src="/blog/images/cyborg-woman_450x246.png" title="Cyborg woman" class="center halfsize liImg2 rounded shadow" alt="Cyborg woman" /> </picture> </div> <p> I said that as an undergraduate engineering student in 1975. Now a Japanese researcher has made <a href="https://news.bbc.co.uk/1/hi/sci/tech/4714135.stm" target="_blank" rel="nofollow">real progress</a> towards a synthetic companion. The artificial skin doesn't seem to have a high density sensory grid that other researchers are experimenting with (<a href="https://news.bbc.co.uk/1/hi/sci/tech/4154366.stm" target="_blank" rel="nofollow">future artificial skins</a> might incorporate sensors not only for pressure and temperature, but also for light, humidity, strain or sound), but I"m sure it won't be long before a similar android with tactile response and <a href="http://www.ai.mit.edu/projects/sociable/videos.html" target="_blank" rel="nofollow">expression ability</a> graduates from the lab. </p> <p> I still want to be involved in bringing that girl... and a boy... and a whole range of socially acceptable companions to market. Some day soon... </p> <p> Update: <a href="https://en.wikipedia.org/wiki/QRIO" target="_blank" rel="nofollow">Sony&rsquo;s Qrio</a>, a humanoid robot under development is exactly along the lines I&rsquo;ve been dreaming of! </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/sonyQrioRobot_225x300.webp" type="image/webp"> <source srcset="/blog/images/sonyQrioRobot_225x300.png" type="image/png"> <img src="/blog/images/sonyQrioRobot_225x300.png" title="Sony Qrio robot" class="center quartersize liImg2 rounded shadow" alt="Sony Qrio robot" /> </picture> </div> <p> Update: from PC Magazine, &ldquo;<a href="https://www.wearable.technology/" target="_blank" rel="nofollow">Eleksen</a>, a small UK-based firm is introducing electronic fabric, essentially carbon-embedded nylon sandwiched between layers of nylon mesh that, when a milliamps charge is passed through it, can recognize touch, pressure and even the direction and path of a stroke. This thin, flexible, and washable fabric connects to a small 8-bit processor, which then can be connected to a standard electronic device like an iPod. Eleksen company executives said the <a href="https://www.sciencedirect.com/topics/engineering/smart-fabric" target="_blank" rel="nofollow">washable fabric</a> can also withstand extreme pressure; they&rsquo;ve rolled a car over it without any ill effects.&rdquo; Seems we are moving towards artificial skin! </p> <p> Update: from The Economist, <a href="https://economist.com/world/asia/displaystory.cfm?story_id=5323427&amp;no_na_tran=1" target="_blank" rel="nofollow">Japan&rsquo;s humanoid robots &ndash; Better than people</a>:<br /> &ldquo;What seems to set Japan apart from other countries is that few Japanese are all that worried about the effects that hordes of robots might have on its citizens. Nobody seems prepared to ask awkward questions about how it might turn out. If this bold social experiment produces lots of isolated people, there will of course be an outlet for their loneliness: they can confide in their robot pets and partners. Only in Japan could this be thought less risky than having a compassionate Filipina drop by for a chat.&rdquo; </p> Self-Fullfilling Prophecy 2005-05-15T00:00:00-04:00 https://mslinn.github.io/blog/2005/05/15/self-fullfilling-prophecy <h2 id="heisenberg">Heisenberg&rsquo;s Uncertainty Principle</h2> <p>The Greek philosopher Democritus believed that the natural world was immutable. In 1927 <a href="https://www.aip.org/history/heisenberg/p01.htm" target="_blank" rel="nofollow">Werner Heisenberg</a> published his &ldquo;Uncertainty Principle&rdquo; which implied, amongst other things, that observing a quantum phenomenon also significantly altered the phenomenon. For many people, such an idea was counter-intuitive. </p> <p> What is true in the physical realm is even more true in the realms of interpersonal relationships and business. Our beliefs and belief systems affect outcomes more than we perhaps realize. Consider the diagram below. </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/prophecy1.webp" type="image/webp"> <source srcset="/blog/images/prophecy1.png" type="image/png"> <img src="/blog/images/prophecy1.png" title="First prophecy" class="center halfsize liImg2 rounded shadow" alt="First prophecy" /> </picture> </div> <h2 id="trends">Describing the Trend</h2> <p> Without knowing what the diagram represents, can you postulate the trend for the next cycle or two? The answer is, of course, no. You need to have an internal model of the behavior of the system and the entity&rsquo;s behavior within that system represented in the graph before you can make a prediction. Without any additional information, you might anticipate the trend like this: </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/prophecy2.webp" type="image/webp"> <source srcset="/blog/images/prophecy2.png" type="image/png"> <img src="/blog/images/prophecy2.png" title="Second prophecy" class="center halfsize liImg2 rounded shadow" alt="Second prophecy" /> </picture> </div> <p> Your model is based on incomplete information, and is necessarily a simplification of the actual system. If you are told that the graph depicts rainfall, and if you believe in a raingod, then your natural reaction might be to find a virgin to sacrifice. If you instead had the belief that rainfall had diminished due to clearcutting of a vast forest which no longer acted as a natural storage mechanism for rainwater, your reaction might be to start lobbying for an intensive tree planting effort. Both belief systems have at their core a central belief that action can affect the outcome to some degree, so your prediction as to how much rain will fall in subsequent years will rely on the vigor with which countermeasures are applied, and your belief in their efficacy. Probably your belief would be that rainfall levels will tend to vary around historical norms, like this: </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/prophecy3.webp" type="image/webp"> <source srcset="/blog/images/prophecy3.png" type="image/png"> <img src="/blog/images/prophecy3.png" title="Third prophecy" class="center halfsize liImg2 rounded shadow" alt="Third prophecy" /> </picture> </div> <p> What if you were told instead that the first graph represented sales of a hybrid car model, and that &lsquo;experts&rsquo; believed that increasing SUV sales and the economy were believed to be responsible for declining sales? A marketing campaign that addressed the rising cost of gas and promoted decreasing the reliance of the US economy on foreign oil as being truly patriotic might gain traction. One might attribute the initial sales spike as being due to purchases by early adopters. The tapering off of sales might be the result of tax incentives that artificially promoted the sales of vehicles over 6,000 pounds. </p> <p> It is clear that China&rsquo;s consumption of gas will soon outpace even the US&rsquo;s rate of consumption, so gas prices will continue to rise, and remain at higher prices than any we have experienced to date. The US Department of Energy produced the following <a href="https://www.eia.doe.gov/emeu/cabs/china.html" target="_blank" rel="nofollow">projection</a> of Chinese gas consumption in June 2004. </p> <p> In 2003, BP showed <a href="http://www.gasandoil.com/goc/features/fex32825.htm" target="_blank" rel="nofollow">China's new influence on global oil supply</a>: &ldquo;Global oil demand, meanwhile, was broadly flat, increasing 290,000 bpd to 75.7 mm bpd from 75.5 mm bpd. All of the increase is attributable to China where oil consumption increased 5.8 % or 332,000 bpd.&rdquo; As demand for a commodity increases, prices increase. As China continues to dramatically increase its demand for oil, prices can only continue higher. In a <a href="http://www.gasandoil.com/goc/features/fex51961.htm" target="_blank" rel="nofollow">feature article</a> on <a href="http://www.gasandoil.com/goc/features/" target="_blank" rel="nofollow">Alexander&rsquo;s Gas &amp; Oil Connections</a>, John Vidal writes &ldquo;China's oil consumption, which accounted for a third of extra global demand last year, grew 17% and is expected to double over 15 years to more than 10 mm bpd &ndash; half the US's present demand. India's consumption is expected to rise by nearly 30% in the next five years.&rdquo; Jim Meyer, an expert at London's Oil Depletion Analysis Centre was quoted in <a href="http://www.gasandoil.com/goc/features/fex51962.htm" target="_blank" rel="nofollow">an article last month</a> as saying &ldquo;After 2007, we can&rsquo;t see enough new supplies to meet almost any reasonable level of demand growth.&rdquo; Expect gas prices to permanently exceed $50/barrel, and $100+/barrel will be reached with startling speed. With that in mind, a long term sales trend for hybrid cars in the US might be projected to look like this: </p> <div style="text-align: center;"> <picture> <source srcset="/blog/images/prophecy4.webp" type="image/webp"> <source srcset="/blog/images/prophecy4.png" type="image/png"> <img src="/blog/images/prophecy4.png" title="Forth prophecy" class="center halfsize liImg2 rounded shadow" alt="Forth prophecy" /> </picture> </div> <p> This projection assumes that historical sales figures are not a significant factor when gauging future sales of this hybrid car model, and that factors on the market that the product sells into have not yet been fully felt or internalized by consumers. </p> <h2 id="belief">Self-Fullfilling Prophecy</h2> <p> What if a hard-nosed automotive executive, looking at ROI on a quarter-by-quarter basis decided that the return was not immediate enough, and terminated the hybrid car based on sales history to date? That automotive manufacturer would soon miss the world&rsquo;s largest market, China, as the demand for fuel-efficient cars will soon be supported by a Chinese middle class newly able to afford more expensive hybrid cars. Unless the manufacturer was somehow able to introduce another technology, such as hydrogen powered vehicles, and successfully put together a national infrastructure network to support hydrogen fuel stations, they will have traded off short-term profitability for long-term viability. </p> <p> Returning to the original idea behind this blog posting, one&rsquo;s expectations and beliefs do color what one might think to be reasonable. If you believe that most people are kind and generous, your behavior will probably reinforce this belief &ndash; with the result that even when you meet Scrooge incarnate, he might crack a smile despite himself. If you think that sales of product XYZ are going down, down, down and that nothing can be done about it then that product is as good as dead. If instead, you come up with an alternate world view, communicate higher expectations and provide sufficient resources over the appropriate time period, that product&rsquo;s best years may be yet to come. </p> Marshall Brain is a very smart man 2005-04-03T00:00:00-05:00 https://mslinn.github.io/blog/2005/04/03/marshall-brain-is-very-smart-man <p> He founded <a href='https://howstuffworks.com/' target="_blank" rel="nofollow">How Stuff Works</a>, and wrote many interesting and informative articles for the site. </p> <p> We share an interest in personal robotics. Like many US citizens, Mr. Brain's focus in Robotic Nation is on the potentially negative aspects of a integrating robots into society, such as mass unemployment. His writing reminds me of the endless fear-mongering that CNN's Lou Dobbs serves up daily on the evils of outsourcing and illegal immigration. I have a more optimistic point of view on personal robotics. </p> <p> The Japanese attitude for robotics is quite positive, and their enthusiasm for personal robots resembles their enthusiasm for advanced cell phones. iMode mobile phone systems have features beyond what is known in North America, and Japanese society has evolved to incorporate behavioral modes that depend on the advanced features of the iMode phone network. Beyond mobile phones, the Japanese culture celebrates electronic gadgetry, and they already accept robots, including personal robots, into their daily experience. </p> <p> Change is coming, and we can either embrace the future or bemoan the loss of the past. I think personal robotics will become a plaything of the rich in 2015; Mr. Brain predicts the mass market will explode in 2030. It's not like we didn't have any warning. </p> <p>I think I need to start learning Japanese.</p> Top 12 Klingon Programmer Sayings 2005-04-02T00:00:00-05:00 https://mslinn.github.io/blog/2005/04/02/top-12-klingon-programmer-sayings <style> ol { list-style-type:none; counter-reset:item 13; } ol > li { counter-increment:item -1; } ol > li:before { content:counter(item) ". "; } </style> <ol> <li>Specifications are for the weak and timid!</li> <li>This machine is a piece of GAGH! I need dual Opteron processors if I am to do battle with this code!</li> <li>You cannot really appreciate Dilbert unless you've read it in the original Klingon.</li> <li>Indentation?! &mdash; I will show you how to indent when I indent your skull!</li> <li>What is this talk of &lsquo;release?&rsquo; Klingons do not make software &lsquo;releases.&rsquo; Our software &lsquo;escapes,&rsquo; leaving a bloody trail of designers and quality assurance people in its wake.</li> <li>Klingon function calls do not have &lsquo;parameters&rsquo; &mdash; they have &lsquo;arguments&rsquo; &mdash; and they ALWAYS WIN THEM.</li> <li>Debugging? Klingons do not debug. Our software does not coddle the weak.</li> <li>I have challenged the entire quality assurance team to a Bat-Leth contest. They will not concern us again.</li> <li>A TRUE Klingon Warrior does not comment his code!</li> <li>By filing this PTR you have challenged the honor of my family. Prepare to die!</li> <li>You question the worthiness of my code? I should kill you where you stand!</li> <li>Our users will know fear and cower before our software! Ship it! Ship it and let them flee like the dogs they are!</li> </ol> <p> I lifted this from Monsters Island, now defunct. </p> Zample in a Blog Post 2005-01-05T00:00:00-05:00 https://mslinn.github.io/blog/2005/01/05/zample-in-blog <p> Erik Dasque, Product Manager for <a href="https://mono-project.com/" target="_blank" rel="nofollow">Mono</a> at <a href="https://www.novell.com/linux/ximian.html" target="_blank" rel="nofollow">Ximian</a> (now part of Novell) was the first person to include a Zample in a blog post. Way to go, Erik! May there be many more bloggers using Zamples to display live code in their blogs. </p> <p> I have had some trouble doing this &ndash; when I used SnipSnap it simply wouldn't accept raw HTML. I wrote a SnipSnap plugin to allow raw HTML, but then my SnipSnap installation b0rked and I gave up on it. WordPress really mangles the HTML for a Zamples button; here is my best effort (can't get the 'Try It!' button to remain vertically centered.) </p> <table> <tbody> <tr style="vertical-align: middle;" valign="middle"> <td><form action="" class="formInput" method="post" target="TomcatMain"> <input name="cmd" type="hidden" value="// Define a runnable C# class here. You must include a // &quot;main&quot; method. Command line arguments go in the text area above. public class Test { public static void Main() { System.Console.WriteLine(&quot;I love Zamples!&quot;); } }"> <input name="imports" type="hidden" value="# Command line arguments go here"> <input name="applet" type="hidden" value=""> <input name="format" type="hidden" value="4608"> <input name="pre" type="hidden" value="true"> <input name="stylesheet" type="hidden" value="/assets/css/style.css"> <input class="formInput/" type="submit" value="Try It!"></form> </td> </tr> </tbody> </table> <pre data-lt-active='false'>public class Test { public static void Main() { System.Console.WriteLine("I love Zamples!"); } }</pre> <p>If you would like to include a live code example in your blog, first create it using the Zamplizer (easiest if you click on one of the selections under the Create Sample left-hand menu on the Zamples Solutions page &ndash; that will set up the Zamplizer with a short working program in the language you want, which you can then modify.)</p> <div style="text-align: center;"> <picture> <source srcset="/assets/images/zamples.webp" type="image/webp"> <source srcset="/assets/images/zamples.png" type="image/png"> <img src="/assets/images/zamples.png" title="Zamples logo" class="center liImg2 rounded shadow" alt="Zamples logo" /> </picture> </div> Another Software Expert Assignment 2004-12-24T00:00:00-05:00 https://mslinn.github.io/blog/2004/12/24/another-software-litigation-assignment <p> I&lsquo;ve just picked up another software litigation case, this time doing research for a software vendor prior to going to court. My client is concerned that two former employees used their code in a product sold by a new company that the former employees started. </p> <p> I've worked on this type of case before. They are often settled out of court. It is interesting to see how human interactions affect technology and business decisions. </p> <p>I know what I'll be doing over the Christmas/New Year's holidays!</p> Nabu cast a long shadow 2004-12-18T00:00:00-05:00 https://mslinn.github.io/blog/2004/12/18/nabu-cast-long-shadow <p> Twenty years ago an Ottawa-based startup was formed by the merger of nine Canadian companies. Nabu sold a cable set-top box with a computer that booted off the cable for under $1000. I was one of the early employees. </p> <p> Nabu was first in many ways, but it never became the commercial success that the founders had hoped. Many years have since passed, and I forgot about the experience. In the last couple of years I have been contacted by various people regarding Nabu&rsquo;s products and services, mostly former customers. </p> <p> &ldquo;Nabu was the first computer that I had when I was a kid. I remember playing games like Mummy&rsquo;s Tomb, Kiddy Park etc. Talk about being ahead of it's time! ... I think maybe people that were influenced by NABU when they were young are probably at the point in their life where they can stop and take a look around... Anyway, I thought I would drop you a message and say that this Ottawa boy (who now lives in Seattle, WA) still remembers the old Nabu days. I&rsquo;m now a professional video game developer, which I&rsquo;m sure was influenced by playing the Nabu so long ago. Of course today&rsquo;s machines have a little more power! What you guys did was a pretty amazing accomplishment IMHO.&rdquo; </p> <p> &ldquo;As someone who was fortunate enough to have access to the NABU network in my after school program at Regina Public School in Ottawa... I really enjoyed it as a kid.&rdquo; </p> <p> Recently I was contacted by a legal firm that has retained me to assist them in a patent litigation case. They are going to cite some of Nabu&rsquo;s products as prior art. I&rsquo;m having fun going back and reconnecting with people that I knew from that bygone era, who I lost track of. So far no-one that I knew has died of old age, which pleases me. </p> Live Code Examples for JDK 6 (Mustang) 2004-12-09T00:00:00-05:00 https://mslinn.github.io/blog/2004/12/09/live-code-examples-for-j2se-6-mustang <p> No sooner did Sun push J2SE 5 (also known as JDK 1.5) out the door, then talk of the next version began. Java releases are generally 18 months apart, but for the first time Sun is letting the world see development builds as they are created. </p> <p> We at Zamples took advantage of this new openness and proudly bring you Live Code Examples for J2SE 6. Let me know if you discover any cool new features! </p> <div style="text-align: center;"> <picture> <source srcset="/assets/images/zamples.webp" type="image/webp"> <source srcset="/assets/images/zamples.png" type="image/png"> <img src="/assets/images/zamples.png" title="Zamples logo" class="center liImg2 rounded shadow" alt="Zamples logo" /> </picture> </div> All Global 2000 Businesses Are Software Developers 2004-11-04T00:00:00-05:00 https://mslinn.github.io/blog/2004/11/04/web-services-means-all-global-2000 <p> With the advent of web services, any company that wishes to expose a programmable interface to their business processes also inherits the issues faced by software vendors. Connecting up a supply chain isn't quite as simple as one might want it to be. </p> <p> Software vendors have an urgent need for their channel partners and customers to adopt their technology quickly. The faster their products are understood, the more money they make. Shorter adoption times drive top line revenue. </p> <p> Similar dynamics affect companies that use web services to transact business. Businesses with custom interfaces need to somehow show system integrators how to program to their web services. Executives need to be able to wire their enterprises together quickly and easily in order to take advantage of business opportunities. Web services are supposed to make this easy... but nothing is ever quite as easy as it is supposed to be. </p> <p> The answer for both software vendors and businesses with custom web services interfaces is a comprehensive developer relations program. Not suprisingly, live online code examples are an important aspect of such a program. I have begun writing a follow-on to my article entitled <a href='https://productmanagement.buzz/index.php/2004/11/10/top_10_issues_f/' target="_blank" rel="nofollow">Top Ten Issues for Developer Relations Managers</a>. I would be happy to hear from you if you would like to review a draft before it is published. </p> Reading Java Properties Files from Bash 2004-11-03T00:00:00-05:00 https://mslinn.github.io/blog/2004/11/03/reading-java-properties-files-from-bash <p>Ever notice that Java properties files are fairly similar to bash scripts? Here is a short bash script that takes a valid Java properties file called <tt>language.properties</tt>, massages it into a format that is compatible with bash syntax, writes it to a temporary file and sources it.</p> <pre data-lt-active='false'>#!/bin/bash TEMPFILE=$(mktemp) cat language.properties | \ sed -re 's/"/"/'g | \ sed -re 's/=(.*)/="\1"/g' &gt; $TEMPFILE source $TEMPFILE rm $TEMPFILE</pre> <p>After running this script, all of the properties in the Java properties file become available as environment variables.</p>