To make a portable application, you just have to package everything into a single directory (which, itself, can contain sub-directories), and then use relative paths. Make sure to include all dependencies you might need, as you can't count on them being installed on the machine that the application is running on. For save-games and settings, you want to store anything that follows the user inside that directory (say, so that they can put the game on a USB drive and continue on a different machine), and any machine-specific settings or data in someplace like %appdata% -- say, video quality settings. In general, portable software is also written to assume that the user has no special privileges to the machine; this is good practice for all software, but is necessary for portable apps, as the user may not even have temporary admin rights, say, to install necessary dependencies like DirectX.
U can go one step further and make your application a single file (it is not necessary). If you wanted to do that, you'd need to write all your resources onto the end of your .exe file, and then take special care to read them back appropriately. Usually you'd first package the resources into something like a .zip file, and then append that file to the end of the executable; hen you can treat the zip file (which may or may not be compressed) as a sort of file system. This is kind of a neat approach, but its a fair bit of work and complication, without providing much benefit over the single directory approach.