Sources for Antique and Unusual Fruits
Well, I promised you I'd find you nursery sources for these sort of things. The first thing that struck me is that it's a heck of a lot easier to find heirloom apples than anything else. Second, is that it's next to impossible to find good sources for the older small fruits. Anyway, here's what I've got (some of these places I've dealt with, others not...most I've at least heard good things about, but use your own judgement):
Trees of Antiquity
This is one of the few places with heirlooms in a full range of fruit species. Like most, they've got a wide range of apples, but there's also got pears, stone fruits, figs, nuts, strawberries (a few), grapes, etc. They've got a big selection and a nice website. Be aware that not quite everything is truly "heirloom" (though most of it is), so if that's important to you, you might want to do a little research.
Vintage Virginia Apples
I've been linking to this operation's website for descriptions of apple cultivars. They deal exclusively in apples, and they've got one of the broadest selections I've seen. The website offers a photo of the fruit of each cultivar, along with a short description and history.
I've been very pleased with this operation in the past, and it's worth getting the catalog just as interesting reading material. Not an heirloom fruit nursery, but a great source (sometimes the only source) for unusual fruits. In addition to a pretty decent line up of the standards, they also feature such unusual treats as aronia, salal, saskatoon, goumi, jujube, and sea buckthorn, as well as many others. Unusual species of the major fruits are also available, such as Fragaria moschata and Fragaria vesca strawberries. They also are one of the few sources for the intergeneric Sorbus hybrids I mentioned earlier, 'Shipova' and 'Ivan's Belle'. My wife and I have given Raintree gift certificates as wedding gifts for our last several weddings, and so far every one has been very pleased.
Just Fruits and Exotics
Located just south of Tallahassee, Florida, this place has a stunning array of species, well-adapted to the sourthern climate. Lots of unusual fruits, and a number of local heirlooms they've gone out of their way to preserve. A great source for those in warmer climates. Friendly folks, and a fun place to visit.
There are lots of others out there...if there's anything you can think of that I'm missing, let me know and I'll add it to this post.
If all else fails, the USDA's germplasm repository system may be able to help. If it's still around, and it's not under patent, they'll probably have it. Please keep in mind, however, that this is not a nursery--material is very limited, and legitimate researchers get first priority. Still, they're usually willing to share with the general public if all other options have been exhausted (don't ask them for stuff you can get through commercial sources if you can help it).