Contributor and SMX speaker, Brooke Osmundson, explains how machine learning is changing our account work and why we need to be smarter about layering our campaign assets in 2020.
Below is the video transcript:
Hi, my name is Brooke Osmundson and I am the associate director of research for NordicClick Interactive. And today I want to talk about the top things that marketers should focus on the most for 2020.
The first thing I want to talk about is machine learning. It’s no secret that it is part of our lives right now, and part of your jobs. But what I’m thinking is going to happen is, it’s going to shift your focus on what you’re doing day-to-day within your accounts. We’re going to see less tactical pieces that we have to focus our time on. And it’s really going to help you be more strategic in your account. So with machine learning, what data can give you, what can it do for you to free up more of your time to start thinking more bigger picture and focus on those bigger picture decisions.
The second piece I want to talk about is audience layering on top of your campaigns with the differences in search or match types. You know they’re kind of not a thing anymore, so we’ve got to be smarter about layering on the assets that we have available to us in our campaigns in order to really reach our right customer based on what we know about them.
More predictions for 2020
Opinions expressed in this article are those of the guest author and not necessarily Marketing Land. Staff authors are listed here.
About The Author
Brooke Osmundson serves as the associate director of paid search at NordicClick Interactive with over six years’ experience. She helps her clients grow their digital strategies using tactics from paid search, social media and programmatic marketing. With her experience and passion in analytics, strategic planning and everything digital, she helps create relevant customer experience strategies at every stage of the user funnel. Brooke has been featured in the Journal of Digital & Social Media Marketing, spoke at SMX West and PubCon Pro, contributed to NordicClick’s 2018 US Search Award for “Best Use of Video in a Search Campaign” and most recently had her client strategy shortlisted for two 2019 US Search Awards. When not working, you can find her enjoying a round of golf, catching up on the latest episodes of Law & Order SVU or completing never-ending house renovations.
I want to find out what happens when we bring machine learning to cheap, robust devices that can have all kinds of sensors and work in all kinds of environments. And I want you to help. The kind of AI we can squeeze into a US $30 or $40 system won’t beat anyone at Go, but it opens the door to applications we might never even imagine otherwise.
Specifically, I want to bring machine learning to the Arduino ecosystem. This has become recently possible thanks to improvements in hardware and software.
On the hardware side, Moore’s Law may be running out of steam when it comes to cutting-edge processors, but the party’s not over when it comes to microcontrollers. Microcontrollers based on 8-bit AVR processors dominated the Arduino ecosystem’s early years, for example, but in more recent years, embedded-chip makers have moved toward more powerful ARM-based chips. We can now put enough processing power into these cheap, robust devices to rival desktop PCs of the mid 1990s.
On the software side, a big step has been the release of Google’s TensorFlow Lite, a framework for running pretrained neural networks—also known as models—on so-called edge devices. Last April, IEEE Spectrum’s Hands On column looked at Google’s Coral Dev Board, a single-board computer that’s based on the Raspberry Pi form factor, designed to run TensorFlow Lite models. The Coral incorporates a dedicated tensor processing unit and is powerful enough to process a live video feed and recognize hundreds of objects. Unfortunately for my plan, it costs $150 and requires a hefty power supply, and its bulky heat sink and fan limit how it can be packaged.
But fortunately for my plan, Pete Warden and his team have done amazing work in bringing TensorFlow Lite to chips based on ARM’s Cortex family of processors. This was great to discover, because at my open-source hardware company, Adafruit Industries, our current favorite processor is the 32-bit SAMD51, which incorporates a Cortex-M4 CPU. We’ve used the SAMD51 as the basis for many of our recent and upcoming Arduino-compatible boards, including the PyGamer, a simple battery-powered gaming handheld. What if we could use it to literally put AI into people’s hands?
Warden had created a speech-recognition model that can identify the words “yes” and “no” in an analog audio feed. I set about seeing if I could bring this to the PyGamer, and what I might do with a model that could recognize only two words. I wanted to create a project that would spark the imagination of makers and encourage them to start exploring machine learning on this kind of hardware.
I decided to make it as playful as possible. The more playful something is, the more forgivable its mistakes—there’s a reason why Sony’s artificial pet Aibo was given the form of a puppy, because real puppies are clumsy and sometimes run into walls, or don’t follow instructions.
I recalled the original Tron movie, where the hero is stuck in cyberspace and picks up a sidekick of sorts, a single bit that can say only “yes” or “no,” with an accompanying change of shape. The PyGamer has a 1.8-inch color display, with 192 kilobytes of RAM and 8 megabytes of flash file storage, enough to display snippets of video from Tron showing the bit’s “yes” and “no” responses. The PyGamer’s SAMD51 processor normally runs at 120 megahertz, which I overclocked to 200 MHz for a performance boost. I connected an electret microphone breakout board to one of the PyGamer’s three JST ports.
Then I turned to the trickiest task: porting the TensorFlow Lite ARM code written by Warden and company into a library that any Arduino programmer can use (although not for every Arduino board! Even as a “lite” framework, the RAM requirements are far beyond the 2 kB of the Arduino Uno, for example).
I found the source code well written, so the biggest challenge became understanding how it handles incoming data. Data is not digested as a simple linear stream but in overlapping chunks. I also wanted to expose the code’s capabilities in a way that would be familiar to Arduino programmers and wouldn’t overwhelm them. I identified the functions most likely to be useful to programmers, so that data could be easily fed into a model from a sensor, such as a microphone, and the results outputted to the rest of the programmer’s code for them to handle as they wish. I then created an Arduino library incorporating these functions, which you can find on Adafruit’s Github repository.
Putting it all together, I wrote a short program using the new library. Now when I press a button and speak into the PyGamer’s attached microphone, the appropriate Tron clip is triggered if I say “yes” or “no,” letting me walk around with my own animated sidekick.
Although this project is a limited (but fun) intro to machine learning, I hope it persuades more makers and engineers to combine AI with hardware explorations. Our next steps at Adafruit will be to make it easier to install different models and create new ones. With 200 kB of RAM, you could have a model capable of recognizing 10 to 20 words. But even more exciting than voice recognition is the prospect of using these cheap boards to gather data and run models built around very different kinds of signals. Can we use data from the PyGamer’s onboard accelerometer to learn how to distinguish the user’s movements in doing different tasks? Could we pool data and train a model to, say, recognize the sound of a failing servo or a switching power supply? What surprises could lie in store? The only way to find out is to try.
This article appears in the August 2019 print issue as “Making Machine Learning Arduino Compatible.”
Editor’s note: Limor Fried is a member of IEEE Spectrum’s editorial board.