You are viewing a single comment's thread.

view the rest of the comments →

MadWorld ago

You could use httrack to mirror a website, preferably on the top of vpn with lower flow-control setting. It should not give you too many "Too Many Requests" stat.

dicedtomatoes55-2 ago

It didn't really help, just gave me back the things you see on the site (including the fonts and such) - that much could easily be figured out by right clicking on the page itself and going to "inspect".

MadWorld ago

I think the pen results you got is somewhat misleading. Many urls shared in your scribd list are dead. That can mean two things:

  1. Those files were already cleaned up.
  2. Or the pen test relied on the most commonly used words (a dictionary) to probe for directories/files and report the stats.

The problem with the latter is that an invalid url path could be interpreted as forbidden (code 403), with implication that the path itself actually exists. So if you go to https://www.cometpingpong.com/pay-to-play-pizza/kids/, it may give you the "403 Forbidden" stat code, which showed up on most of your links. It does not mean that there is actually a directory named pay-to-play-pizza on that website; it only means that you did not have the permission to poke this url path. I think this is something worth considering, before jumping to the conclusion or following down an empty path...

Of all the links in your list, may I ask which urls were valid?

Thanks!

@Vindicator, this may interest you.

SearchVoatBot ago

This comment was linked from this v/pizzagate comment by @think-.

Posted automatically (#14107) by the SearchVoat.co Cross-Link Bot. You can suppress these notifications by appending a forward-slash(/) to your Voat link. More information here.

dicedtomatoes55-2 ago

This sounds really plausible. But why would the two scans find different directories if the second option was the case ("Or the pen test relied on the most commonly used words (a dictionary) to probe for directories/files and report the stats.") Wouldn't the results be the same for each?

I have noticed that some that are in the results for Comet Ping Pong are not in the results for Buck's Fishing and Camping and vice versa.

The only url's that were completely valid off the list were the ones that you see and can click on when you go to the website.

MadWorld ago

This sounds really plausible. But why would the two scans find different directories if the second option was the case ("Or the pen test relied on the most commonly used words (a dictionary) to probe for directories/files and report the stats.") Wouldn't the results be the same for each?

The results would be different, if the pen test only runs a subset of the dictionary words. This may have to do with speeding up the pen test. Remember, it is a pen/probe test, it does not have to run at every combination.

I have noticed that some that are in the results for Comet Ping Pong are not in the results for Buck's Fishing and Camping and vice versa.

The only url's that were completely valid off the list were the ones that you see and can click on when you go to the website.

I would say running the pen test may be useful and speedy, but you should manually verify that the links are good, that the links actually point to meaningful files. It is not good enough to use the 403 code as an indicator. And when you do find a valid file, use the archive.fo to preserve its content. For some tricky websites that do not let you archive the links, you could use proxy sites to indirectly archive the sources. I have used this method several times for this purpose.

dicedtomatoes55-2 ago

Even though it says 403 or 429, it could lead somewhere?

MadWorld ago

429 means you are making too many requests and the site rejects the requests.

403 means the site has configuration in place, that forbids you for probing/discovering the directory path.

For the 429 code, you would have to try it at a slower pace, to see if it returns any other code. For the 403 code, you cannot know if the directory is valid or not. If you hit a valid file, however, you can still validate its existence. For example, http://cometpingpong.com/good-directory/ may give you a 403 code, but the http://cometpingpong.com/good-directory/pizza.php may give you a proper file. And this is how you verify the directory good-directory itself is actually valid. But without hitting a valid file, you may never know. Another exception is if the site is misconfigured to show directory listing, where all files are listed, simply by navigating to the directory in question.

To answer your question, yes, it could lead to somewhere. But that is unlikely to give you the desirable outcome. But if you have a good list of words in your dictionary, you could probe it through proxies, and do it gradually over a long period of time.

dicedtomatoes55-2 ago

Wow, thanks for the lesson. Good to know all this going forward. But, another question. For the 429's, what could be making all of the requests?

MadWorld ago

Wow, thanks for the lesson. Good to know all this going forward.

Happy to help! I should be thanking you for doing PG research!!! You may never know, but someday you could be the one saving those unfortunate kids!!

For the 429's, what could be making all of the requests?

This has to do with the pen test website that sends out too many requests within a short duration. For example, it sends out 20 URLs to probe for with in a few second. Websites do not like that and it could be perceived as an attack. It is also not very nice to drain a website's resource by making too many requests, because the site has to serve many users/customers, not just your pen test.

Thank you!!

dicedtomatoes55-2 ago

So a combination of my test plus customers/users is making those pages that currently say "too many requests" unusable. How could those specific pages once again become usable or when would I be allowed entry (if this is the case and I'm not reading all of this wrong).

telleveryoneyouknow ago

The bottom line really is to leave this to the professionals so you don’t get hurt.

Either that or study up on some non cp leads where you can practice and get proficient

MadWorld ago

The bottom line is, the website does not like any user making too many requests. Maybe you could try testing one url per 1 to 5 seconds. I am not sure if you have any control of this parameter, if tested through that pen tool website. If you do have this option, you could just rerun your pen test at a slower pace.