This is the first article in a short series about the different components of AXElements. AXElements has been rewritten to be much more modular and now various components may be used individually. The keyboard event generator was the first component to be polished and released as its own gem.
Event generation is exposed via the Accessibility::String mix-in. Using the mix-in, a simple string of human readable text is taken and turned it into a sequence of events which may be fed to one of the keyboard event posting APIs provided by OS X.
To get started you will need to have a working Ruby 1.9
implementation—I’ve tested this with MRI 1.9.3 and MacRuby (for
MacRuby you will need a nightly build).
Then you can install the
UPDATE: The gem has been reported to not compile if your Ruby was compiled with GCC. The preferred solution would be for you to recompile Ruby with Clang or use MacRuby. 😝 I’m not a pro with C extensions, but I’ll look into whether or not I can force the extension to compile with Clang for you people who do not want to, or cannot, recompile with Clang.
UPDATE 2: AXTyper-0.7.4 now checks if Ruby’s CC is Clang. If it is not Clang then it tries to find Clang in your $PATH. A proper error message is given if Clang cannot be found. Clang is binary compatible with GCC, so this should be safe to do.
The quickest demonstration of event generation is through
1 2 3 4 5 6 7 8 9 10 11 12 13 14
And now you have some events! Events are number/boolean tuples; the number is a key code and the boolean is the key state; there are more details regarding the key code, but I’ll get to that later.
Now that we have some events, we should figure out what to do with them since they are not of much use on their own. The most useful, and easiest, thing to do is post them to the system. This of course comes with a caveat: you will actually be causing the keys to be typed out to the system and the front most application will receive the events.
Be careful of the string you give to the generator!
The interface for posting events is different depending on if you are
using MRI or MacRuby. With MacRuby, events can be posted natively in
Ruby code, but for this demonstration the C extension provided by
AXTyper includes a singleton method,
takes a single event and posts it to the system. Using this method we
can post our events:
1 2 3 4 5 6 7 8
You’ll notice that typing begins immediately when the
method is called. Typing is also really fast; in fact, it has to be
slowed down for practical use.
Let’s add a wrapper to do generation and posting together, we’ll
alsoadd a small time buffer so that you get a chance to release the
return key after entering a command in
1 2 3 4 5 6
Now let’s try something a bit more complicated:
A Little More Depth
So what’s going on here? On the AXElements side of things there is just a basic lexer and generator. The lexer tokenizes the string input and the generator takes the tokens and generates one or more pairs of event tuples. The interesting detail is that a set of escape sequences for control keys has been added.
Strings fed to the lexer support an extra set of escape sequences for
the control keys like Control, Option, Command, etc.. The last example
above used the
"\\CONTROL" escape sequence to represent the left
control key. Other than those special sequences, string formatting
is straight forward and you can use all the letters, numbers, and
symbols that you would in any other string. Uppercasing letters is
automatically handled for you and so are all of the symbols that you
would have to hold down the shift or option key in order to
type. White space and line breaks in strings will get turned into
tabs, spaces, and return/enter, and delete key presses appropriately,
but you can also use
"\b" as you
would in any other string.
Once the lexer is done lexing, its output is fed into the generator.
The event generator inspects each token and figures out what events need to be generated in order type the token.
As mentioned above, events are number/boolean tuples. They are fed to the CGEventCreateKeyboardEvent() function that is used in the above examples. The function takes a key code and a key state as parameters and so each event pair is simply what is required to be passed to that function.
Key codes are a mapping of numbers to keys on the keyboard; some mappings are static, such as the control keys; and some mappings are dynamic based on the keyboard layout. Since a code refers to a physical key and not the particular symbol they represent, upper case letters require the generation of events for pressing either the shift key before hitting the lower case letter.
The key state is simply a boolean value with
true meaning that the
key is in the keydown state and
false meaning that the key is in the
keyup state. This makes it possible to hold down the shift key and
then press the
"a" key to generate an
"A"; it also makes it
possible to simulate hotkeys and symbols.
Symbols such as
"]" are all supported by the event
generator. Any symbol that you can type directly using one or more
keys should Just Work™. Some contrived examples might look like
1 2 3
A caveat to this is that some symbols can be on the keyboard twice, the obvious cases are the mathematical operations and numbers. The event generator does not use the keypad for plain symbols in strings. To use the keypad keys specifically you will need to use custom escape sequences.
As mentioned above, standard escape codes (e.g.
"\t") all still
work, even using
"\b" as the delete key. On top of the built in escape
sequences, the lexer and generator have added an additional set of
escape sequences for control keys and other static keys. The full list
is located in the documentation
but the naming convention should be obvious enough after a small
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
There are two rules to using the custom escape sequences. First, a custom escape sequence must be terminated by an empty space or the end of the string.
The second rule is an exception to the first rule; a custom escape
sequence can end with a
"+" if it is being used in a hot key
combination and will be immediately followed by another key in the
combination. In previous examples we combined two and three keys in
order to make a combination, but an upper limit on the number of keys
is not defined by the event generator.
Sending Events To A Specific Application
Though the easiest way to post events to the system was to use
CGEventPost(), there are also APIs for posting events to a specific
application regardless of which application is focused.
The API for doing this is located in the OS X Accessibility headers, but requires you to provide a reference to the application where you would like to post events. Using the accessibility APIs is a little tricky at first, especially in Ruby.
I’ll cover Accessibility API basics in the next part of this series: "Accessibility: From References To Objects".