iOS Custom Views and Accessibility in Table Views

Part of the genesis for this article starts with http://ronnqvi.st/making-drawrect-accessible/, which encouraged me to try and improve the accessibility of my Greek Interlinear Bible app. It also led me down into the depths of iOS's Voice Over mechanism -- a place, frankly, I really don't ever want to visit again! But for those who need it, it's worth it.

The other part of the genesis for this article lies simply with the fact that my app needs a lot of labels in a lot of table cells. And by “lot,” I mean on the order of a hundred or more per cell. Perhaps an example would help:

This is a good representation of a typical table cell in my app. By my very quick count, there are 128 words, each of which needs to be positioned separately. (You could arge that the English text on the right could be typeset in one label. But for my purposes, that's not an option.)

When I first wrote the app, pre-1.0, I used UILabels. Makes sense, right? These are labels, after all, so why not use the framework component that most makes sense. Except there was a problem: performance.

There's two places in the app that have to worry a lot about performance:

• Typesetting: The text itself is parsed and then typeset. While fast for everything that's going on inside, it still takes a good deal of time to process. More than what we can do in the time needed to achieve smooth scrolling when tableView:cellForRowAtIndexPath: is called, anyway.
• Creating all those labels: In order to keep table cell displays reasonably quick, we have to create all the labels, separate of their cells.
• Assigning labels to the cell: To actually display our pre-computed labels, we have to assign them to the cell.

In pre-1.0, things weren't as optimized as they are now (and there's still a long way to go there), but the long-and-short of it is that the typesetting only computed where the labels went -- it didn't actually create them. After the typesetting was complete, we looped over all the labels in the verses in a chapter and created all the UILabels for them. And then when it came time to display a cell, we removed all the subviews and then added each label back to the cell.

And it was slow.

I mean, really slow. Not quite-unusable slow, but so slow that I started wondering how in the world this could be slower than similar code done in Javascript and HTML for the prototype!

Turns out that UILabel, while great for what it does, doesn't do so well in large numbers -- like one or two thousand of them. The time drain comes particularly when they are created, but they also cost in another way: every time we presented a cell, we had to delete the previous labels and add in the new ones. Both were very time consuming. [I suppose we could have built every table cell at the start and ignored the re-usable cell method that iOS uses. But we'd still have had a slow time when creating the labels and assigning them to each cell.]

Two solutions were born to help deal with the performance:

• Create a light-weight label that we could use in place of a UILabel.
• Create those labels up-front during the typesetting phase.

These labels were called PKLabels, in a stroke of naming brilliance, and supported the very basic UILabel properties like text, font, various -Color properties, etc. Over time it also grew a couple additional properties to support what the app needed.

But there were several key differences between our label and a real label:

• Super-fast initialization. UILabel has to go through surprisingly a lot to create itself. PKLabel doesn't.
• No drawing mechanism. UILabel knows how to draw itself -- but PKLabel doesn't. Instead, we can create a custom UITableViewCell that has a labels property and knows how to display these labels as fast as it possibly can.
• No accessibility.

The first two were great: the app realized great performance improvements at the huge cost of the latter.

Perhaps one could argue whether accessibility is a great need in an app like this -- a lot frankly depends upon the visual layout, and frankly, how does one make a view like this particularly friendly to users who need the accessibility options? [A bit more egregious, however, is the general lack of usability in some of the more popular Bible apps on iOS when using VoiceOver. Some try, and then some simply... don't.]

Version 1.0 and 1.1 slapped a bandage on the problem, and dumped the text of both the left and right side into the cell's accessibilityLabel property. Which worked, but also meant that VoiceOver users couldn't use parts of the app, like tapping-and-holding on a word in order to bring up a context menu.

It's been something I've wanted to change ever since 1.0, but couldn't come up with a great way to do it. And then I read the article at http://ronnqvi.st/making-drawrect-accessible/ and thought, “That might just work!”

And, while I was at the top of the table view, it did. Beautifully. I could tap a word, and VO would select it, say it, and then let me perform gestures on it.

And then I scrolled the view, and it all fell to pieces.

It turns out that the code in the article just doesn't work when it's in a table cell. Why? It boils down to the way one has to create the UIAccessibilityElement's frame -- it has to be screen coordinates, not view coordinates. Which worked perfectly as long as the table wan't scrolled -- the screen coordinates that were calculated for each label during drawRect: matched what was actually displayed. But the moment a scroll occurred, drawRect: wasn't called -- unless a new cell was generated. This meant that the screen coordinates passed to VoiceOver no longer matched the screen, and so both it and I were terribly confused.

What to do? I searched far and wide and finally figured a few things out. First: we need to recalculate the element's accessibilityFrame whenever iOS calls the overridden method to get a particular element, and Second: we have to recalculate everything when a scroll occurs.

First, remember that custom table cell that I mentioned? That's where all the hard work has to be done, since it's the owner of all these custom labels. It's the only thing that knows how to draw them, and as such, is the only thing that can calculate the accessibility frame.

But there's a fly in the ointment: if we do this on drawRect:, we end up creating UIAccessibilityElements every time the cell needs to be redrawn -- which is slow. We can do better -- and the answer is to create these elements once: when the table cell gets all the labels assigned to it (which still occurs every time the cell comes into view):

-(void)setLabels:(NSArray *)labels
{
  _labels = labels;
  accessibilityElements = nil;
  accessibilityElements = [[NSMutableArray alloc] initWithCapacity:_labels.count];
  for (int i=0;i<_labels.count; i++)
  {
    PKLabel *theLabel = _labels[i];
    UIAccessibilityElement *ae = [[UIAccessibilityElement alloc] initWithAccessibilityContainer:self];
    ae.accessibilityFrame = [self convertRect:theLabel.frame toView:nil];
    ae.accessibilityLabel = theLabel.text;
    ae.accessibilityTraits = UIAccessibilityTraitStaticText;
    [accessibilityElements addObject:ae];
  }
}

Calculating the frame here is really of little importance -- because we already know the frame will be invalidated once the table view scrolls. Nevertheless, it's here for completeness. By the way, accessibilityElements is an NSMutableArray, just in case you were wondering.

Next, we need to override the AccessibilityElement methods. The first,isAccessibilityElement is easy: we just return NO:

- (BOOL) isAccessibilityElement
{
  return NO;
}

Then we need to return the number of custom labels in our cell. There's an additional fly in the ointment (the ointment is starting to get crowded, isn't it?): we may have real views in this cell -- and they need to be handled correctly too!

- (NSInteger) accessibilityElementCount
{
  if (accessibilityElements)
    return [accessibilityElements count] + [self.subviews count];
  else
    return [self.subviews count];
}

Next, we need to override accessibilityElementAtIndex:, which iOS uses to return an accessibile element. Note that since we have to handle both our custom labels and our view's subviews, things get... hairy:

- (id) accessibilityElementAtIndex:(NSInteger)index
{
  NSInteger aCount = 0;
  if (accessibilityElements)
    aCount = accessibilityElements.count;
  if (index < aCount)
  {
    if (accessibilityElements)
    {
      UIAccessibilityElement *ae = accessibilityElements[index];
      [self _recalculateAccessibilityElementByIndex:index];
      return ae;
    }
    else
    {
      return nil;
    }
  }
  else
  {
    NSInteger subViewIndex = index - aCount;
    if (self.subviews.count>0)
    {
      if (subViewIndex < self.subviews.count)
      {
        UIView *aView =  self.subviews[subViewIndex] ;
        return aView;
      }
      else
      {
       return nil;
      }
    }
    else
    {
      return nil;
    }
  }
}

And finally, we override indexOfAccessibilityElement:. Again, we have to handle the possibility that we have subviews, so we search the accessibilityElements array first, and if nothing is found, we search our subviews.

- (NSInteger) indexOfAccessibilityElement:(id)element
{
  NSInteger anIndex = NSNotFound;
  if (accessibilityElements)
  {
    anIndex = [accessibilityElements indexOfObject:element];
  }
  if (anIndex == NSNotFound)
  {
    anIndex = [self.subviews indexOfObject:element];
    if (anIndex != NSNotFound)
    {
      anIndex += [self accessibilityElementCount];
    }
  }
  return anIndex;
}

Now, the most important piece of all of that is one line in accessibilityElementAtIndex::

[self _recalculateAccessibilityElementByIndex:index];

This function recalculates the UIAccessibilityElement's frame when called. It does essentially the same thing as when we created the element, but it's just worried about the frame. And this is critical: without it, our frames would be out-of-sync with the screen.

Except that as we scroll the table view, we're still out-of-sync. Why, why, why?

It turns out that views like UILabel are built to recalculate their frames when scrolling happens. But our custom labels aren't part of the view hierarchy, and so they don't get that notification. It turns out we need to add just a little bit of code to the tableViewController in order to make things better in -scrollViewDidScroll:: (based on http://stackoverflow.com/a/8209674/741043)

for (id cell in self.tableView.visibleCells)
  if ( [cell accessibilityElementCount] >0 )
    for (int f = 0; [cell accessibilityElementAtIndex:f]; f++);

This finishes everything off -- by calling accessibilityElementAtIndex, we force all the custom labels (and views) in the cell to recalculate their accessibilityFrame, and VoiceOver is happy again.

Now, it's not all perfect. It seems that if the user misses any UIAccessibilityElements, VoiceOver tries to pick the nearest one. Which works great on a navigation bar. But for some reason, it doesn't work so great here -- it just seems to pick a random word in the cell. There's no real rhyme or reason as to what it picks, and that's just a little frustrating. Perhaps by expanding the frame by a few pixels on either side will help by making the targets easier to tap, but there are still plenty of blank areas in the cell where you'd like VoiceOver to do the right thing.

And another fly in the ointment (somebody change the ointment, please!): when we're running without VoiceOver, things have slowed down so much as to make scrolling really herky-jerky.

The reason is obvious: we're going through all those labels and views at the end of every scroll, and we're creating all those UIAccessibilityElements every time a cell comes into view (which, frankly, is the more obvious problem, because it happens during the scroll).

For a user who needs VoiceOver, I'm willing to sacrifice the performance here: being accessible is more important that scrolling in a perfectly smooth manner. But the remainder of the app's users will definitely notice the performance degredation. What to do?

To help that problem, I check UIAccessibilityIsVoiceOverRunning() to see if we even need to worry about creating the UIAccessibilityElements and if we need to calculate their frames. This brings scrolling performance back in line when VoiceOver is disabled, but allows the app to create those elements when VoiceOver is enabled.

One last fly in the ointment (last fly, I promise): If VoiceOver is turned on after the app has already rendered the table view and the cells and labels therein, VoiceOver has no UIAccessibilityElements to work with. I haven't come up with a solution for that just yet -- I assume there's a notification I could latch on to that would let us take care of that, but I haven't got that far yet. I'm not sure it's a normal use case, either -- do users frequently turn VoiceOver on and off if they really need it? Probably not, but I could be wrong.

So there you have it. It was a long journey over many frustrating hours, but we got there. It's not perfect (far from it), but VoiceOver users can now access all the same functionality a non-VoiceOver user could. And we maintain performance for the non-VoiceOver users too. While we may not be doing things in drawRect: anymore, like the article that kicked this all off, we're very much still doing things in the spirit of the article, with just a few tweaks to make it work well in something that scrolls.

So -- what about your apps? If you want to use any of this code, it's MIT licensed, so have at it. You'll have to adjust it to suit your needs, I'm sure, but it should give a good start. If you want to see some of the additions the app uses that I didn't show here (because they aren't really related), the app is on GitHub (https://github.com/photokandyStudios/gbible; look in https://github.com/photokandyStudios/gbible/blob/master/gbible/PKTableViewController.m#L192 and https://github.com/photokandyStudios/gbible/blob/master/gbible/PKTableViewCell.m) and licensed under CC-BY-NC-SA. The relevant code to this post, however, is MIT licensed.

And if I've missed something extremely obvious and the way I'm doing things is obtuse (or wrong, even), let me know. I'm no expert in iOS's accessibility framework, and if there's a better way, I'd love to hear it.