is it true if i change my hosting server, it will damage my rank on search engines?
Does domain transfering effects search engine optimization?
Its absolutely a myth. Your domain name ranks and not the ip on which it is hosted.
when I switched my site, it went down in rank for like 2 days, then after Google spidered it, it was right back to where it was.
As far as I know, Nothing of that kind will effect on your SERPs. But if your site is targeting a local version SERPs of Google (say Google.co.uk) and your old hosting was based in the UK and the new hosting server is located in some other country (other than UK), most probably it would affect your Local Google SERPs.