HTML Sitemap is for your user only and so you can do whatever you think would be good for your user but XML Sitemap is submitted to Google and so you need to follow the format.