\documentclass{acmsiggraph}                     % final
%\documentclass[annualconference]{acmsiggraph}  % final (annual conference)
%\documentclass[review]{acmsiggraph}            % review
%\documentclass[widereview]{acmsiggraph}        % wide-spaced review
%\documentclass[preprint]{acmsiggraph}          % preprint

\usepackage[scaled=.92]{helvet}
\usepackage{times}

%% The 'graphicx' package allows for the inclusion of EPS figures.
\usepackage{graphicx}
%% use this for zero \parindent and non-zero \parskip, intelligently.
\usepackage{parskip}
%% Optional: the 'caption' package provides a nicer-looking replacement
%% for the standard caption environment. With 'labelfont=bf,'textfont=it',
%% caption labels are bold and caption text is italic.
\usepackage[labelfont=bf,textfont=it]{caption}

%% If you are submitting a paper to the annual conference, please replace 
%% the value ``0'' below with the numeric value of your OnlineID. 
%% If you are not submitting this paper to the annual conference, 
%% you may safely leave it at ``0'' -- it will not be included in the output.
\onlineid{0}

\title{Neural Network System for Eyebrow Morse Code Decoding}
\author{Charles Lohr\thanks{e-mail: lohr1 at umbc.edu}\\ University of Maryland, Baltimore County }

%% Keywords that describe your work.

\keywords{morse code, neural net, ann, hci, alternative input, head band}

%%%%%% START OF THE PAPER %%%%%%

\begin{document}

%\teaser{
%  \includegraphics[width=1.5in]{sample.eps}
%  \caption{Lookit! Lookit!}
%}


\maketitle


%% Abstract section.

\begin{abstract}

Morse code is an effective way of enabling a human-interactive data
stream with a minimum amount of auxiliary information.  It consists of
nothing more than dots, dashes, and pauses of varying lengths. This
asynchronous stream of data is both able to be interpreted and created
by humans at high speeds, ranging from 35 to 70 word per minute~\cite{Pierpont02}.

Neural Networks have, in several cases been utilized to interpret human
inputted Morse code ~\cite{Yang07} and ~\cite{Luo02}.  These studies
have a varying range of method of input.

We examine the possibility of Morse code input through a headband system~\cite{Torsten07}
to enable text input for people with disabilities in this paper.

\end{abstract}

%% ACM Computing Review (CR) categories. 
%% See <http://www.acm.org/class/1998/> for details.
%% The ``\CRcat'' command takes four arguments.

\begin{CRcatlist}
  \CRcat{H.5.1}{Information Systems and Presentation}
{User Interfaces}{Alternative Input Devices};
  \CRcat{I.5.1}{Pattern Recognition}{Models}{Neural nets}
\end{CRcatlist}

%% The ``\keywordlist'' command prints out the keywords.
\keywordlist

\section{Introduction}

%% The ``\copyrightspace'' command must be the first command after the 
%% start of the first section of the body of your paper. It ensures the
%% copyright space is left at the bottom of the first column on the first
%% page of your paper.

 \copyrightspace

There are many people who are unable to move large portions of their body,
many times this prevents them from being capable of interacting with 
a computer without special equipment.  Potential disabilities can include impaired
hands, limbs large portions of the lower body's motor control, and inability to control
speech.  Special equipment such as eye trackers are many times unavailable to people
with serious disability due to extremely high costs in the range of several thousand dollars.

Research has been done in the area of alternative input through a headband system \cite{Torsten07}.  We build on this
input system by improving it strictly in the arena of text input.  By limiting ourselves
to text input, we are able to stay within the standard domain of Morse code.

We enable, in a single session training of two artificial neural networks (ANNs for short)
the recognition of two different classes of actions out of a streaming data set (an analog signal stream from the headband).
These two patterns are generally very similar however, have certain distinct features in the
time domain.  Our two classifications are ``Dots" and ``Dashes" because both of which have the
same motion, it was important for us to  get additional information to aide the ANNs.  The similarity in the motion is that they both require a brow furl and brow un-furl.  The difference is in the length of time the brow remains furled.

We partition up the data stream into discrete snapshots - whenever the input data signal exceeds certain bounds
then it triggers an event that can either be used for training or for practice.  If a dot ($\bullet$) or a dash (---) is found
it is then appended to the current "letter."  If the gesture is not identified as either, that is noted.

End-of-letter and end-of-word gestures are indicated simply by lack of input - just as they were in
standard Morse code.  This is not determined by an ANN.


\begin{figure}[t]
\centering
$\bullet\bullet\bullet\bullet \quad \bullet \quad \bullet-\bullet\bullet \quad \bullet-\bullet\bullet \quad - - -$
$\bullet- - \quad - - - \quad \bullet - \bullet \quad \bullet - \bullet \bullet \quad -\bullet\bullet$
\caption{Hello world in Morse code.}
\label{fig:hello}
\end{figure}

\section{Background}

\subsection{Morse Code}
Morse code was invented by Samuel Morse in the 1840s.  It grew to wide
spread use in the telegraph industry because of the narrow communications
path.  The technology at the time only allowed for a single asynchronous
stream.  This technology was adapted in radio to be used as a tone, a single
binary stream that could either indicate a 1 (tone) or a 0 (silence).

Using only this 1 or 0 input signal, dashes (---) and dots ($\bullet$) can be determined by the
length that the 1 is set.  $\bullet$s are intended to be down for 1/3 the length of a Dash.
Letters are formed by a series of ---'s and $\bullet$s that are closely packed together
with a pause (0) equivalent to three $\bullet$s to indicate end of letter.  Words are
separated in a similar way, with a pause equivalent to seven $\bullet$s.  An example
of one such phrase can be found in Figure ~\ref{fig:hello}.

Morse code was effective because of humans ability to quickly understand,
interpret, and communicate in it.  Many times experienced operators describe hearing
entire sentences instead of the tone that makes up the data stream~\cite{Pierpont02}.
All Licensed Amateur Radio operators in the US were required to be able to
understand basic Morse code up until the late 1990's.

\subsection{Data Input}
Torsten Felzer has done much research in the area of human input devices,
recently using an low cost head band, enabling patients to communicate with nothing
more than eyebrow twitch.  The current systems in place largely rely on a patient entering
text into the computer using a grid array of letters and phrases.  The user is required to
wait specified amounts of time in order to trigger the eyebrow device to select a given column
or row.

This enables users to give more complicated commands to the computer and does not limit them to
text entry.  The mechanism for text entry in this system is relatively slow.  We are attempting
to avoid all non-text entry because of the extra time required to provide input.

\subsection{Artificial Neural Networks and Morse Code}
There has been research recently in the arena of using Artificial Neural Networks for
interpretation of Morse code.  In many of the current implementations the ANNs either takes the
Morse code in a standard input, or in more interesting ways that do not relate to conventional
input.  Most of the current research also follows proper rules, i.e. that a dot is supposed to
be 1/3 the length of a dash.

The research has also looked more deeply into additional features such as continual training,
to allow the network to continue to learn how to best understand a user's Morse code input.
This is useful because many times users change their way of expressing $\bullet$s and ---s.

\section{Exposition}

\begin{figure}[t]
\centering
\includegraphics[width=3.5in]{screen1.png}
\caption{A standard run of the program, after training}
\label{fig:standardrun}
\end{figure}

Our system makes use of the same headband used by Felzer and takes a continuous data stream that
is produced at roughly 250Hz and stores it in a buffer - constantly keeping the most up to date
data in the front.  The data is then normalized to +1 and -1.  The data is displayed to the
user for debugging purposes, as can be seen in Figure~\ref{fig:standardrun} \textbf{G}.  Additionally,
there are two bands (\textbf{F}) surrounding the stream, these bands settle to $\pm$ 0.05.  If the input
data stream exceeds these bounds, the bounding will expand outward to encompass the input signal.
Upon a spike (immediate left of \textbf{G}), a notification will be sent to an interface that
handles interesting events.  The bounding bars will settle back to normal over a period of time.

The user should begin by recording training samples, this can be done by pressing
Figure~\ref{fig:standardrun} \textbf{D}'s "Train Dot" or "Train Dash" buttons.  Once clicked, at every
event the program feels is a possible training sample, it will record it in the bank to the right.
If recording a $\bullet$, the training sample will go in \textbf{A}.  If the record is a ---, it will go
in \textbf{B}.  If a training sample is erroneous, the user can take note of it by looking at the bank.
The user can then re-record all samples in a given bank, thus making them all exemplars for the final
output.  When recording samples \textbf{E} will say "Training" and when waiting for input it will
read "Waiting".

Once the user is happy with all of their training samples, they can then click Figure~\ref{fig:standardrun}
\textbf{D}'s Program button.  This will invoke the creation and training of two ANN's.  After two or three
seconds, the network is programmed and will automatically be placed into Normal Mode.  At this time, the
network is ready for input.

For debugging purposes, Figure~\ref{fig:standardrun} \textbf{H}, the current output from the --- ANN as
well as \textbf{I}, the output from the $\bullet$ ANN are presented to the user.  This enables feedback
to the user as to what they could do to improve results.  Note that the output of these networks
only is only examined 100ms after a peak is found.  A basic understanding of this can be seen in
Figure~\ref{fig:dotdash} -- you can see when the system samples the two networks at the orange dash.
You can see that for a $\bullet$, red is significantly higher than green and for a ---, green
significantly higher than red.  Green refers to the output of the --- network and red, the output of the $\bullet$ network.

Once the system finds out if the user is doing a $\bullet$ or a ---, it will append to the display the
indicated input event to Figure~\ref{fig:standardrun} \textbf{C}.  Once a substantial wait has accumulated,
this will be cleared and the line will be printed to the terminal.

In the event the system is unable to discern if the event was a $\bullet$ or a ---, a ; will be placed in
the stream.  This could be used at a later time to help the system "Guess" what the user ment by looking
at surrounding contextual information.

\begin{figure}[t]
\centering
\includegraphics[width=2.8in]{screen2.png}
\caption{A diagram of a $\bullet$ (top) and --- (bottom).}
\label{fig:dotdash}
\end{figure}

\section{Artificial Neural Network}
The architecture of this neural network is a very basic feed-forward network that is designed to take
samples from the input stream, sampled and averaged every 16 samples, and operate on that.  It gets
a series of snapshots of an input data stream in the time domain.  It then has a hidden layer of
nine nodes.  This all feeds into an output node.  A diagram of this can
be seen in Figure~\ref{fig:nn}.

The network contains 9 hidden nodes in a single layer.  One of the nodes in the hidden layer
and one of the nodes on the output layer are 
Linear type node as seen in Equation 1, the rest are all a standard sigmoid curve function
as seen in Equation 2.


\begin{equation}
 F(x) = x
\label{eq:lin}
\end{equation}


\begin{equation}
 F(x) = \frac{ e^{ x * 2 } - 1 } { e^{ x * 2 } + 1 }
\label{eq:sig}
\end{equation}

The network topology is identical to a standard feed-foward network, so any systems that already implement
this form of an ANN can immediately take this formation on. 

Originally everything was placed into a single ANN with two outputs, however we found that the system performed
better as a whole if there were two separate networks, one for the $\bullet$ and one for the ---.

\begin{figure}[]
\centering
\includegraphics[width=2.9in]{diagram.pdf}
\caption{A diagram structure of one of our ANNs}
\label{fig:nn}
\end{figure}

\subsection{Training}
I chose to use standard back-propagation.  Because we have several training samples of exemplar data and can easily produce many cases
of non-exemplar data, it is possible for us to train the networks using back-propagation.  All exemplar data, for one type of input is
associated with a positive desired output, zero-value vectors are fed in with a negative desired output, and the opposite training data
is fed in as an opposite desired output.  A typical training for $\bullet$ would be: +.7 for all of the $\bullet$ training samples, -.2 for
an empty training sample and -.7 for all of the --- training samples.

These values are fed in to the learner which cycles for 30,000 epochs with a weight of .02.  Once this is complete, typical outputs for the
training data is very close to the actual desired output.  For matching patterns, the output value is typically +.5 to +.7 and for 
the opposite patterns, output is typically -.2 to -.4.  Thus showing that the network does in fact accept the training data.

\section{Conclusion}
We were able to get the ANNs to converge to very close to the desired values on learning.  Because of this, we were able to
successfully to tell the difference between a $\bullet$ and a --- with a high reliability.  This enabled our app to output
$\bullet$ and --- information to the terminal interactively, and thus allow the user to input the Morse code.

We feel that ANNs provide a useful and powerful tool for pattern recognition in situations like these, and that their use in this
arena could be expanded.  Because the size of the ANN can remain small, it is possible to perform all calculations easily in real time.

\section{Future Work}
There are many fields for future exploration in this arena.  In the future one should investigate the application of
more techniques used in other Morse code neural networks papers, such as using an ANN to learn how long spaces and end-of-letter pauses should be.  One could
also explore the possibility of having the ANN continue learning after the initial run to adapt to the user as they
change their mode of input.  When users become tired or simply shift what they are comfortable with for $\bullet$s or ---s, this
would not have an adverse effect on the network.

We also want to explore more variations of ANNs and more closely examine ~\cite{Luo02}'s work for possible applications.

We would like the have our application interpret the Morse code and translate it into standard text.  This would greatly
increase the power of this tool, and could make it possible to navigate a computer using nothing more than this headband.
We would also like to implement a system that could send key presses so a user could use a terminal.

Because our system also notes when it finds symbols that it thinks are important but cannot differentiate between a $\bullet$ and a ---,
one could perform a higher order system to "guess" the symbol to complete filling in a word correctly.

\section*{Acknowledgements}

We would like to thank Anjali Reed for help in facilitating this research, acquiring the equipment and source for ideas; Torsten Felzer for providing the equipment at no cost and Dr. Yun Peng.

\bibliographystyle{acmsiggraph}
\nocite{*}
\bibliography{paper}
\end{document}


%\begin{equation}
% \sum_{j=1}^{z} j = \frac{z(z+1)}{2}
%\end{equation}

%\begin{figure*}[ht]
%\centering
%\includegraphics[width=2.5in]{sample.eps}
%\caption{Sample illustration.}
%\end{figure*}
