Searching in Contacts app doesn't search all fields

Specifically, I find that the Contacts app on the iPad won't search the notes field (or the phone number or email fields for that matter).

Question: Is there a setting to enable this or a solution to cause the Contacts app search mechanism to include fields other than names?

It seems reasonable to desire search results to include notes you've made on the contact, phone numbers, emails etc. Often I find myself trying to find a contact based on a note or other info without remembering their name.