You can insert these web pages to the file to get explicitly disregarded. Robots.txt files use one thing known as the Robots Exclusion Protocol. This website will simply produce the file for you with inputs of internet pages to be excluded. There’s a range of glitches that can have an https://rankingseotoolz.com/robots-txt-generator