InftyReader

I recently ran into the problem of needing to read some books on algorithms. I had several books available in pdf and html formats and a couple as hard copies that I scanned manually.
The problem with all these books was that they had a lot of math symbols and equations that I could not get either fine reader or kurzweil to OCR properly. All the math equations and symbols will either come out garbled or not at all.
Pranav pointed me to the InftyReader at
http://www.inftyproject.org/en/index.html
This is a software for math OCR and also for math reading by the blind from a Japanese research institution.
The results from InftyReader were significantly better for me than both fine-reader and kurzweil.
Although I am still experimenting with the trial version, the output I have so far is definitely sufficient for my purposes.
The only drawback is the price tag of $900, which I am sure I do not want t

paper for the CSI conference

The following is the draft of the paper I am sending for the CSI national conference for infomration and communication technologies. This is pretty much a summary of my recent blogs. Will be great if you can provide any comments:

Accessibility with Web 2.0
Manish Agrawal
Introduction
The Web browser has moved on from being a window to static content in the form of html pages on Web sites. Web applications are becoming increasingly dynamic and as rich and interactive as desktop applications with the improvements in available bandwidth, development of client side libraries, Web services and mash ups and so on. All these combined are often referred to as the Web 2.0. These next generation Web 2.0 applications bring with them a new set of accessibility challenges. This paper presents some of these challenges along with the proposed solutions and frameworks being developed to overcome these hurdles.
The purpose of this paper is to increase the awareness about Web 2.0 accessibility challenges and to give companies and developers the information that will help them select the right platforms and tools to make the Web 2.0 applications produced by them accessible by the differently abled.

What are the accessibility challenges
Currently the way most screen readers provide access to Web pages is through the Document Object Model (DOM). The screen reading software loads the entire Web page into a buffer and then provides it as audio output as the user navigates around the browser window.
Some of the accessibility challenges with the current Web pages include:
• Screen reader users are presented a Web page linearly from the top left to the bottom right. This causes them to have to scroll a lot before they can get to the relevant content on a page. Simple mechanism to avoid this like the use of headings to highlight the beginning of various sections of a page, proper tab stops, assigning shortcut keys to frequently used options etc. are not frequently implemented.
• Images are used on Web pages without proper alternate text. Often these images are clickable and act as buttons allowing users to perform various actions. These kinds of user interface elements are completely inaccessible for screen reader users.
• Tables and CSS are incorrectly used in a way that the screen reader software is unable to correctly read out page contents.
In addition to these and other existing problems, Web 2.0 applications present a new set of challenges. Some of these include:
• Rich content may be presented in a container that does not expose accessibility hooks required by screen readers. Some common examples include WebEx online meetings, Citrics over the Web etc.
• Dynamic HTML allows screen elements like fly-out menus and tree views to be created that can be operated only with the use of a mouse. These screen elements do not have any keyboard alternatives and are completely unusable by screen reader users.
• The use of AJAX allows small areas of a Web page to be updated with new content from the server without reloading the whole page. Although this is good for performance and interactivity of a Web page, it is very difficult for a screen reader to detect and present this new information to the user.
• Plain AJAX and DHTML pages are now being replaced by pages created using specialized libraries and toolkits like the Google Web Toolkit, Microsoft Silverlight, Adobe Flash Flex and so on. All these libraries and toolkits either do not expose any accessibility information for screen readers or have their own proprietary mechanism for providing this information. Screen reader support for applications developed using these frameworks is currently very basic to non-existent.

The next section describes the key solutions being developed for making the rich Web 2.0 more accessible.

Accessibility frameworks for the rich Web
This section lists some of the upcoming standards and frameworks aimed at addressing the issues of accessibility in Web 2.0.

User Interface Automation
User Interface Automation or UIA is the new Microsoft accessibility framework that replaces Microsoft Active Accessibility (MSAA).
While MSAA was meant for exposing accessibility related information using COM, UIA is based on .Net. At a high level, UIA looks like a fairly well thought out framework that addresses the shortcomings of the MSAA world and will hopefully simplify the utterly complex mechanisms that current accessibility technologies have to implement.
The added focus on the use of UIA for automated testing will probably help to push it higher on the priority list of third party control developers - who traditionally haven't either looked at accessibility or only looked at it as an after thought.
UIA will be implemented by all Microsoft products, including MS Office, Internet Explorer etc.

IAccessible2
IAccessible2 is a framework similar to UIA but is still based on COM. This has been created by the open Linux foundation and builds on top of the work done by Microsoft on MSAA. . The purpose of IAccessible2 is to fill some key gaps in MSAA related to how accessibility related information should be exposed by applications specially related to spreadsheets and tables, complex documents etc.
IAccessible2 is being adopted by large vendors like IBM in their office suite and open source projects like Firefox. The standard has been synchronized with the UNIX/Linux GNOME Accessibility Toolkit (ATK).
Using this framework, existing screen readers can provide richer information to the users with the MSAA support already built in. This framework, however, does not provide a good answer for accessibility in applications built using the .Net platform.

Accessible Rich Internet Applications (ARIA)
ARIA is the upcoming W3c recommendation/standard for the accessibility of rich internet applications.
So, how does ARIA translate into accessibility?
There are basically three pieces to the accessibility puzzle:
1. ARIA: provides a standardized markup that browser based rich application UI libraries can implement to describe the UI and provide standard events for dynamic updates.
2. The browser: While rendering the apps that have been marked up using ARIA, the browser translates ARIA tags into the platform specific accessibility API. In the case of Windows, this API may be UIA or IAccessible2. Firefox already implements this translation and IE 8 has announced support for ARIA to UIA translation.
3. Assistive applications like screen readers: These are the clients of applications that expose accessibility related information through UIA, MSAA or IAccessible2 and help people with disabilities use those applications.

ARIA is gaining a lot of acceptance in the browser and UI toolkit vendors. Support for ARIA is being built into Microsoft Silverlight 2.0, Google Web Toolkit, Dojo from IBM and so on. These frameworks are still under development and Adobe flash and some smaller third party control developers have still not caught on to this.

AxsJAX
AxsJAX is a JavaScript framework that helps inject accessibility features into Ajax Web applications so that users of adaptive technologies such as screen readers and self-voicing browsers experience the same level of interactivity that is now taken for granted by users of Web 2.0 applications.
AxsJAX injects accessibility enhancements as defined by W3C ARIA. The prerequisites for experiencing its benefits include:
• A modern Web browser like Firefox 2.0 or later that supports W3C ARIA.
• Adaptive technologies that respond correctly to the accessibility enhancements introduced by W3C ARIA.
• In particular, many of the enhancements injected by AxsJAX depend on support for live regions a feature that enables adaptive technologies like screen readers and self-voicing browsers deal correctly with asynchronous updates to portions of a Web page.

The AxsJAX framework can inject accessibility enhancements into existing Web 2.0 applications using any of several standard Web techniques:
• As a bookmarklet --- small snippets of JavaScript that are used to create smart bookmarks.
• Using Greasemonkey --- a powerful browser extension that allows end-users to customize the look and feel of Web sites via custom scripts.
• Using Fire Vox --- Fire Vox, an open source talking browser extension for Firefox, automatically injects the AxsJAX scripts if the "Use site specific enhancements" option is turned on.


Conclusion
The future of the Web is rich and interactive powerful applications. These applications pose significant new accessibility challenges for the users of assistive technologies like screen readers. There are concerted efforts being made by the W3c, software vendors like Microsoft, Google, and IBM, and the open source world to make Web 2.0 more accessible. It is our responsibility as consumers of UI development platforms and frameworks for Web 2.0 applications that we demand these platforms provide the accessibility features required to ensure that the differently abled are not left out of the Web 2.0 revolution.

About the Author
Manish Agrawal is a visually impaired software professional who has over 10 years of extensive software industry experience successfully leading large teams to design and implement complex business-critical applications. Manish is currently working with Sapient Corporation, a leading IT consulting company based out of Boston, USA. Manish is a .NET architect on the Sapient’s Advanced Technology team and writes a blog related to accessibility at http://iaccessible.blogspot.com .

UIA and test automation

This MSDN article about test automation using UIA gives a good overview of how UIA can be used:
Test Run: The Microsoft UI Automation Library

I feel it is too cumbersome for developers to use it the way it is presented in this article. The underlying technology is very useful but will probably need testing automation applications like QTP or silk to start adopting it for large scale use.

ARIA and UI frameworks

ARIA is the upcoming w3c recommendation for making rich internet apps accessible.
I tried looking around for how many rich application platforms/development libraries are building support for this. Here is what I found:
- Silverlight2 has announced support for ARIA this year.
- ASp.net is not talking about ARIA anywhere that I could find. They are sticking with the existing accessibility they have with html...although I would have thought it would have been better with all the interactive features being packed into asp.net controls.
- Flash is still with the MSAA world. I don't know enough about Flash to say if they will move to ARIA or UIA going forward.
- DOJO is an ajax toolkit that is building ARIA support in its widgets. However, my browser hangs every time I try to visit dojo's accessibility page.
- Ruby doesn't seem to be talking about ARIA either. But again I don't know enough about ruby to say that I've looked around enough to state this.

IAccessible2

IAccessible2 is the accessibility API specification created by the Linux foundation.
It builds on top of Microsoft Active Accessibility (MSAA), which was the accessibility API created and promoted by Microsoft in the COM days and untill recently.
Pros:
- MSAA is implemented by all existing assistive technology vendors and it will be straightforward for them to use IA2.
- The API has been harmonized with the Unix accessibility API.
- The API has been implemented in some big projects like firefox and some IBM products.

cons/aprehensions:
- the API builds on top of an old MS API and not on UIA, which is being actively developed and supported by Microsoft going forward.
- I am looking for answers to the following with IAccessible2:
1. The entire specification is for use by Com servers and clients. However, most new application development work on the Microsoft platform is being done in .Net. How does this specification help to make those .Net applications accessible?
2. If .Net/Com interop is the anser, how does this spec address the significant performance overhead of that interop?
3. Are there .Net wrappers to the com API already being created that .Net applications can use without having to implement the com interop themself?
4. Is there a .Net widget/control library that is also being published by the tool vendors supporting this initiative that I can use as a .Net application developer to get out of the box accessibility with my UI elements - as is the case with the UI widgets that ship with Microsoft Visual Studio?
5. If I am developing custom controls/ UI widgets in .Net, are there any sample apps/ prebuilt controls that I can subclass to help make my custom control accessible?

UIA and ARIA

ARIA is the upcoming W3c recommendation/standard for the accessibility of rich internet applications.
So, how does ARIA translate into accessibility?
There are basically four pieces to the accessibility puzzle:
1. ARIA: provides a standardized markup that browser based rich application UI libraries can implement to describe the UI and provide standard events for dynamic updates.
2. The browser: While rendering the apps that have been marked up using ARIA, the browser translates ARIA tags into the platform specific accessibility API. In the case of Windows, this API is now called User Interface Automation (UIA). Firefox already implements this translation and IE 8 has announced support for ARIA to UIA translation.
3. UIA: As mentioned above, UIA is the new accessibility framework for Windows applications that replaces Microsoft active accessibility. It provides a managed interface for accessibility while MSAA was just Com. UIA also simplifies the use of automation across multiple UI frameworks.
4. Assistive applications like screen readers: These are the clients ofapplications that expose accessibility related information through UIA and help people with disabilities use those applications.

UIA and MSAA

User Interface Automation or UIA is the new microsoft accessibility framework that replaces Microsoft Active Accessibility (MSAA).
From the overview at UIA fundamentals, it looks like a fairly well thought out framework that addresses the shortcomings of the MSAA world and will hopefully simplify the utterly complex mechanisms that current accessibility technologies have to implement.
The added focus on the use of UIA for automated testing will probably help to push it higher on the priority list of third party control developers - who traditionally haven't either looked at accessibility or only looked at it as an after thought.

accessible ajax update panel

This useful article about an Accessible ajax update panel doesn't just provide a simple trick to make ajax controls more accessible but will also help developers understand how screen readers interact with Web pages and reiterates my point about using headings.

ARIA

A must read for everyone who wants to know about accessibility of Rich Internet Applications - pretty much everything that we see on the Web today.

automated accessibility testing

Came across this interesting post from last year.
However, no one has left a comment about the RAAKT (The Ruby Accessibility Analysis Kit) that the author is talking about.

I intend to explore a similar tool that ships with Visual Studio.

Headings

Headings - as the name suggests - are used to highlight or mark text appearing at the top of sections of a document.
I want to bring out how invaluable the simple mark up elements h1 h2 etc. are when it comes to web page accessibility using a keyboard.
Imagine that your Web page has about 40 navigation widgets, links and buttons, (not a very large number considering an average application - a random yahoo mail page that I just viewed has 61 links).
Now imagine that using the keyboard, I need to click the 27th item....oops! the poor tester who was denied the mouse is sure to get carpul tunnel tabbing 27 times on every page (a topic I'll touch again in a later post).
This is only the beginning.
Now grasp the fact that a screen reader usually presents a Web page as if someone was reading the page from top left to bottom right linearly with every navigation widget appearing on a new line.
All those top menu links, those left menu items (and often the drop down menus) appear at the beginning.
The relevant content of the current page starts after this - and it is not even a tab stop.
As a screen reader user, I have no simple way of getting to read this "main" content of the page -- something that is visible to a sighted user as the most prominent thing on the screen is accessible only after wading through all that header and left nav clutter.
Try going to the microsoft Msdn pages. The top and left nav clutter there is hundreds of links - I read those pages from the bottom up to avoid this.

Only if the main content would start with a mark up like a heading.... and most screen reader implementations allow me to move through headings in a page

This very basic usability guideline of marking important sections of the screen with relevant headings is almost universally ignored:
google does it on its main search results page but forgot to do it on all other pages like google scholar search, gmail etc.
yahoo has it in the classic mail view but for some reason forgot to add it to the new and improved yahoo beta.

Do you have headings for identifying parts of the screen in your Web application?
Remember that in addition to their traditional use, these simple tags help users navigate through your app.

fly out menus

If you want to use fly out / drop down menus in your Web application, consider the suckerfish family of open source menus:
Suckerfish Dropdowns

and

Son of Suckerfish Dropdowns | HTML Dog

and
Suckerfish Dropdowns - Perciformes!

A simple google search for suckerfish accessible menus will yield these and other useful links for implementing such menus.

Keyboard

I'll start with the most basic requirement for software accessibility for the visually impaired - allow all navigation and functionality through the keyboard.
It is commonly known that blind users use a software called a 'screen reader" that gives them an audio output interface to the computer instead of having to look at the traditional video interface (the screen).

However, there is also a difference in the input interface that blind users use. They use only the keyboard and not the mouse -unless if they have to.
So, if you want to make your software/web sites accessible for the visually impaired, it is not sufficient to just allow screen readers to read the output. It is equally important to allow users to use only
the keyboard for all input and navigation.

This simple usability requirement has a lot of ramifications when it comes to user interface design.
The last time you created those fly out menus on your Web page, did you ever try using them without a mouse?
The last rich client online trading application with a lot of panes for displaying the different parts of your screen. Did you try moving across those panes using the keyboard alone?
The drop down list that posts back the page on selection change. Did you try selecting the third item in that drop down with the keyboard?

The answer to the above and other similar questions is "no" for most applications and Web sites that I've seen - even as a part of the team that was developing it.

For screen reader users, it is essential that they be able to do all input and navigation through the keyboard. And interestingly enough, it is very very simple to test. Simply deny the luxury of a "mouse" to one person in your QA team.
And treat everything that person cannot do as bugs.
this simple test will have you cover over 50% of the accessibility chalenge - the input interface.

Having said that, I need to also add that this simple test may bring out all the accessibility bugs but their solutions may not be simple.
If you're the one trying to get accessibility in, you'll get push back from first the "tester who was denied the mouse", then the user interface designer who designed the screens and navigation flow, the developer who has to do all that "extra work" to implement the new "more complicated" design, the architect who needs to come up with a "new technical solution" for something that developers in his/her team could otherwise do on their own, the project manager who just cannot understand what all this re-work and extra time is all about and so on.

... so, good luck!
The subsequent entries in this blog will deal with specific tips on how to make apps more keyboard friendly.

purpose

As the name suggests, the purpose of this blog is to write about software accessibility for people with disabilities.
I'll start with vision disabilities because that is what I am most familiar with - being a blind technologist.