fix(server): Provide GoogleBot compatibility.

[hackartists]

• Actually, GoogleBot will crawl as below command.

curl -A GoogleBot 'http://localhost:8080'

• However, dioxus render_handler rejects GoogleBot crawl.

  • It DOES mean dioxus webapp can't be indexed by GoogleBot properly.

• I removed accept checking instead of optional parameter.

  • Because all dioxus app based fullstack will be desired to be crawl and proper SEO.

• Before this PR, it only allows requests including accept, however it allows all requests

• I would say that its part doesn't need to provide backward compatibility
  because SEO capability is a crucial issue.

  • It means it can make starters confusion why they can't check their page on
    google inspection.
  • So I propose that it SHOULD provide SEO feature by default.

[jkelleyrtp]

I have been meaning to make this change for a while - it's annoying when trying to test endpoints locally that they deny requests without any content type. I've suspected that it hurts SEO.

LGTM but pinging @ealmloff since he's more familiar with this and why would we want this in the first place.

I think we just wanted better filtering for MIME types, but this code has always been suspect to me.

[ealmloff]

The code that was removed filters out requests if they have a content type, but it is not text/html. If there is no content type, then it should accept requests either way. However, that doesn't handle the case where content type is */* which might be what some scrapers use. I don't think we need to handle the accept content type at all as long as we don't have multiple different handlers on one route (eg. one for json and one for html)