Possible solutions I found while searching for a solution myself:
Check if IIS is installed correctly:
run %windir%\Microsoft.NET\Framework\v4.0.30319\aspnet_regiis.exe –i
or for x64
run %windir%\Microsoft.NET\Framework64\v4.0.30319\aspnet_regiis.exe –i
http://forums.iis.net/t/1149449.aspx
Comment out the Visual Studio httpHandlers from web.config after the commenct
<!–
The Visual Studio Development Server uses these httpHandler configuration values.
To run the server under IIS, comment these handlers so that IIS will use the
duplicate set of handlers in the system.webServer section below.
–>
http://getsatisfaction.com/live_labs_pivot/topics/handler_cxml_has_a_bad_module_managedpipelinehandler_in_its_module_list
Add .cxml as an allowed mime type
Most pages recommended using .cxml and text/xml, but somewhere I also saw .cxml text/cxml recommended. I also set .dzi and .dzc to text/xml.
http://serverfault.com/questions/179591/iis7-how-to-configure-server-to-serve-cxml
Set the correct Application Pool
For me what fixed the problem was setting the correct Application Pool for the site.
It was set to .NET Classic, it has to be on ASP.NET v4.0
http://forums.silverlight.net/forums/p/209710/494967.aspx
Other links that I browsed
http://forums.silverlight.net/forums/p/205826/482545.aspx