I have a selection of large pieces of card (A3 or A2, from the stationery store) or sheet plastic that I use as backdrops, usually just by placing them on a flat surface and against a wall, so that there is an even curve at the right angle between the ground and back. The card I use most often is a large silver-coloured piece, as it can be used to look quite white under strong lighting, but generally just gives a nice gradient of grey that works as contrast against most collectibles.
I've been using the large anti-static foam envelope that came wrapped around someone's new LCD TV (you get smaller versions around iMac computers too). Just use a clip or some sellotape to hold it up against a wall / door / etc. and then adjust the curve using something small and weighty at the front underneath the camera's view. I've also got a few white cardboard boxes that make good stands.
It's also best to use something to stand the camera on rather than holding it. A tripod would be great, but I've just been using a pile of big books of various thicknesses - add more to the pile to raise the camera, take some away to lower it.
Here's a quick pic of my current favoured set-up (and the results)...paired cheap, daylight-balanced (5500K) continuous studio lights, and a sheet of white styrene. All just portable gear that's fairly quick to get out and store away when finished. The fully artificial and controlled lighting means I can take sample data of just the background and use it for custom white balancing.
