Member Avatar for Dani

We use a lot of AJAX that loads JSON files that return an application/json mime type. In some instances, these JSON files are used to assemble elements on the page, so we don't want to hide them from Google because we want Google to see the page fully rendered.

My question is whether we should be adding an X-Robots-Tag: noindex HTTP header to these JSON files. On one hand, we don't want them returned in the search results because they're JSON files. On the other hand, these files are used as building blocks to build the page we do want indexed, and therefore should be included in Google's cache the same way Google's cache includes our CSS and JS files. We wouldn't noindex our CSS, after all. Right?

  • 2 Contributors
  • 1 Reply
  • 58 Views
  • 2 Months Discussion Span
  • Latest Post Latest Post by Chris Hüneke
Member Avatar for Chris Hüneke

Yes, it’s generally a good practice to set JSON files that are not meant to be displayed as standalone content to noindex.

This prevents these files from being indexed, still allowing Google to see and index the rendered content generated by these JSON files. This way, you ensure that users find the relevant content without having irrelevant JSON files show up in search results.

Google is quite good at executing JavaScript and rendering pages fully nowadays.

This means your dynamically loaded content generated by your JSON files should appear in the rendered page and can be indexed, even though the JSON files themselves are set to noindex.

If the content provided by the JSON files is essential for the indexing of your page, the rendering of the complete page by Googlebot ensures that this content gets indexed without needing the JSON files themselves to be indexed.

CSS and JS files contribute to the visual and functional design of your website but are not relevant content for search results.

So it’s fine not to set them to noindex.

JSON files contain data that eventually gets displayed on the page, but the files themselves shouldn’t be what users find in search results.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.