It looks like jQuery Dialog doesn't really have an default enter key to click behavior. This is the kind of thing you would see on OS or Application dialogs. Also, there's not really a defacto way to do this, but this seems to work pretty well:

$('.gps-cancel-form').dialog({
    autoOpen: false,
    resizable: false,
    height:200,
    width:500,
    modal: true,
    title: 'Confirm Navigation',
    buttons: [
        {
            click: function() {
                App.backToOptionsHelper();
                $( this ).dialog( 'close' );
            },
            tabindex: 1,
            class: 'dialog-button',
            text: 'Leave Page'
        },
        {
            click: function() {
                $( this ).dialog( 'close' );
            },
            tabindex: 2,
            class: 'dialog-button',
            text: 'Stay on Page'
        }
    ]
});

$('body').on('keydown', '.dialog-button', function (event) {
    var myself = this;
    if (event.keyCode === $.ui.keyCode.ENTER 
              && $(document.activeElement).hasClass('dialog-button')) {
        $(myself).trigger('click');
        $(myself).blur();
        event.stopPropagation();
    }
}); 

Of note:

1. The dialog constructor with the array of button objects
2. The delegated keydown event attached to the button class
3. && $(document.activeElement).hasClass('dialog-button') which is really the most important part, because you only want the 'click' to happen on the active element.

Unknowns:

Apparently you have to have tabindexes for this to work?

Help:

[Stack Overflow]: http://stackoverflow.com/questions/2522125/jquery-ui-dialog-make-a-button-in-the-dialog-the-default-action-enter-key

[Stack Overflow]: http://stackoverflow.com/questions/967096/using-jquery-to-test-if-an-input-has-focus