its probably because these forums have safeguards to prevent crawlers and spiders from copying anything and everything here.

imo its very rude to use a spider or web tracker to copy a board without permission first.

it wastes bandwidth and slows down the board for everyone else, just so you get a copy of shit youll never read...

or maybe im an asshole.

If you don't mind me askin doogle why the intrest in copying this site? To be honest im no very conforted at the moment when someone I know little about shows a desire to mirror this site because they have not stated there intended use.

Vesp if you desire a copy then best would be to sign into thru CORE then DL and zip all the files. Obviously cleaning the databse of personal information before releasing it publicly.

Sorry if I came off rude but surely you must undertstand the caution behind my words.

Nevertheless, Vesp can attest to the fact that I did discuss this issue with him.

That is true.
Even though I make MySQL backups on a daily basis, and download the attached files weekly, or at least try too... it is still a very good thing to have non-active HTML files with all of the content present for the sake that he pointed out.

NPH, that's exactly what happened. I saw 1.2Gb and was like, WOW. Then looked and it was all the same damn login page! I tried URL capturing and the live, to no avail.

Jboogie, scroll down to see my thanks....





















...................../´¯/)
....................,/¯../
.................../..../
............./´¯/'...'/´¯¯`·¸
........../'/.../..../......./¨¯\
........('(...´...´.... ¯~/'...')
.........\.................'...../
..........''...\.......... _.·´
............\..............(
..............\.............\...

With love, of course.

Why can't you just provide me with the command line params? Fer shits and giggles?

Dongle, hate to say it but i tried everything under the sun and couldn't get the authentication to work.  it works with some forums great, i seriously think it has to do with php.

vesp, you can use bandwidth throttling with winhttrack.  in fact i always use the bandwidth throttling and i limit the amount of sessions open out of kindness cuz i don't want to intrude on a forum's bandwidth too badly.  as far as having it constantly going, i've never tried it.  i'm certain there is software out there that can do that, otherwise perhaps you can create a script that utilizes command link httrack and schedule it daily or something.

but its not something that is as simple as clicking a few checkboxes abd letting it ride out...

Jboogie, scroll down to see my thanks....

your a good sport! hahah! that was actually pretty funny, plus im a sucker for ascii art.

i cant tell you exactly what to do because your system and setup is not identical to mine. your network setting will vary, plus we probably arent using the same version of windows. from what i gather just from playing with this software, the OS makes a difference with this program. mostly it has to do with the network configuration and the addition of IPv6 support in both vista and Win7...

the filters are another essential aspect of making this progy work right. you will need to set up a good deal of filters and have a decent understanding of the Vespiary's file system and setup. utilize boolean phrases (boolean is supported in this progy) to maximize your results.

the first advice i will give you is to setup a filter to EXCLUDE any links with the word "logout"... with your boolean **'s in the right place. so if you UsedTheirFSE you would already know how to right that one.. it look like ' - *logout* ' in your 'scan rules' in the 'set options' box.

then you have the rest of the shit you dont want... frig sample, the shit like the 'help' screen and the other garbage like theme info. if you dont exclude the theme directory, then you have an assload of shit that will get downloaded. so youll want to want to have something like ' -*/*theme*/* ' again, notice the placement of your boolean modifiers. see also that i didnt use the word 'themes', as your modifier would cause the word 'themes' to be included already.

you will have to decide what you do and dont want to mirror, so its impossible for me to create your filters for you without knowing what you want downloaded... and plus i dont know what the folders/threads are named for everything here. im relatively new here and i dont work on this site. i have admin privileges at WD, so i can openly browse the sites folders and i know where and how things are configured.

and the last thing that i will spoon feed you is the configuration of the one-time proxy setup.. though you should have been able to figure it out by reading the tutorial i told you to look at... so again, i will tell you to read this link:
http://httrack.kauler.com/help/CatchURL_tutorial
now, once you get that page read and comprehended, you will notice that you cannot set up the 'capture URL!' proxy. this is hard to explain without knowing what your OS is and its version. the problem is with the IPv6 protocol... the address that HTTrack gives you will bee in IPv6 format if your system is using IPv6. this isnt really bad, but an IPv6 address is not something that you can use with firefox or iE. if your using XP pre-SP3 then you wont have that problem unless you have installed that specific hotfix yourself or something else did. so with XP, you need to go to network connections properties, and then disable or uninstall the IPv6 in the adapter properties, then reboot. you must reboot or the changes will not happen.

now, vista and win7 are another story. i dont know that they can function properly on a WAN without IPv6, but i dont know cause im not using either atm.

hope that helps more... but as they say around these parts 'your mileage may vary'

JB

Thanks for the spoonfeeding, as you put it, Jboogie. I will go ahead and give this a whirl at some point and make sure to not let you have the archive it creates.

Thanks again. Vesp, if it works, I'll put it up an archive to whatever FTP you want me to

but its not something that is as simple as clicking a few checkboxes abd letting it ride out...

Quote from: Dongle on July 21, 2010, 01:55:55 AM

Jboogie, scroll down to see my thanks....

your a good sport! hahah! that was actually pretty funny, plus im a sucker for ascii art.

i cant tell you exactly what to do because your system and setup is not identical to mine. your network setting will vary, plus we probably arent using the same version of windows. from what i gather just from playing with this software, the OS makes a difference with this program. mostly it has to do with the network configuration and the addition of IPv6 support in both vista and Win7...

the filters are another essential aspect of making this progy work right. you will need to set up a good deal of filters and have a decent understanding of the Vespiary's file system and setup. utilize boolean phrases (boolean is supported in this progy) to maximize your results.

the first advice i will give you is to setup a filter to EXCLUDE any links with the word "logout"... with your boolean **'s in the right place. so if you UsedTheirFSE you would already know how to right that one.. it look like ' - *logout* ' in your 'scan rules' in the 'set options' box.

then you have the rest of the shit you dont want... frig sample, the shit like the 'help' screen and the other garbage like theme info. if you dont exclude the theme directory, then you have an assload of shit that will get downloaded. so youll want to want to have something like ' -*/*theme*/* ' again, notice the placement of your boolean modifiers. see also that i didnt use the word 'themes', as your modifier would cause the word 'themes' to be included already.

you will have to decide what you do and dont want to mirror, so its impossible for me to create your filters for you without knowing what you want downloaded... and plus i dont know what the folders/threads are named for everything here. im relatively new here and i dont work on this site. i have admin privileges at WD, so i can openly browse the sites folders and i know where and how things are configured.

and the last thing that i will spoon feed you is the configuration of the one-time proxy setup.. though you should have been able to figure it out by reading the tutorial i told you to look at... so again, i will tell you to read this link:
http://httrack.kauler.com/help/CatchURL_tutorial
now, once you get that page read and comprehended, you will notice that you cannot set up the 'capture URL!' proxy. this is hard to explain without knowing what your OS is and its version. the problem is with the IPv6 protocol... the address that HTTrack gives you will bee in IPv6 format if your system is using IPv6. this isnt really bad, but an IPv6 address is not something that you can use with firefox or iE. if your using XP pre-SP3 then you wont have that problem unless you have installed that specific hotfix yourself or something else did. so with XP, you need to go to network connections properties, and then disable or uninstall the IPv6 in the adapter properties, then reboot. you must reboot or the changes will not happen.

now, vista and win7 are another story. i dont know that they can function properly on a WAN without IPv6, but i dont know cause im not using either atm.

hope that helps more... but as they say around these parts 'your mileage may vary'

JB

but its not something that is as simple as clicking a few checkboxes abd letting it ride out...

Quote from: Dongle on July 21, 2010, 01:55:55 AM

Jboogie, scroll down to see my thanks....

your a good sport! hahah! that was actually pretty funny, plus im a sucker for ascii art.

i cant tell you exactly what to do because your system and setup is not identical to mine. your network setting will vary, plus we probably arent using the same version of windows. from what i gather just from playing with this software, the OS makes a difference with this program. mostly it has to do with the network configuration and the addition of IPv6 support in both vista and Win7...

the filters are another essential aspect of making this progy work right. you will need to set up a good deal of filters and have a decent understanding of the Vespiary's file system and setup. utilize boolean phrases (boolean is supported in this progy) to maximize your results.

the first advice i will give you is to setup a filter to EXCLUDE any links with the word "logout"... with your boolean **'s in the right place. so if you UsedTheirFSE you would already know how to right that one.. it look like ' - *logout* ' in your 'scan rules' in the 'set options' box.

then you have the rest of the shit you dont want... frig sample, the shit like the 'help' screen and the other garbage like theme info. if you dont exclude the theme directory, then you have an assload of shit that will get downloaded. so youll want to want to have something like ' -*/*theme*/* ' again, notice the placement of your boolean modifiers. see also that i didnt use the word 'themes', as your modifier would cause the word 'themes' to be included already.

you will have to decide what you do and dont want to mirror, so its impossible for me to create your filters for you without knowing what you want downloaded... and plus i dont know what the folders/threads are named for everything here. im relatively new here and i dont work on this site. i have admin privileges at WD, so i can openly browse the sites folders and i know where and how things are configured.

and the last thing that i will spoon feed you is the configuration of the one-time proxy setup.. though you should have been able to figure it out by reading the tutorial i told you to look at... so again, i will tell you to read this link:
http://httrack.kauler.com/help/CatchURL_tutorial
now, once you get that page read and comprehended, you will notice that you cannot set up the 'capture URL!' proxy. this is hard to explain without knowing what your OS is and its version. the problem is with the IPv6 protocol... the address that HTTrack gives you will bee in IPv6 format if your system is using IPv6. this isnt really bad, but an IPv6 address is not something that you can use with firefox or iE. if your using XP pre-SP3 then you wont have that problem unless you have installed that specific hotfix yourself or something else did. so with XP, you need to go to network connections properties, and then disable or uninstall the IPv6 in the adapter properties, then reboot. you must reboot or the changes will not happen.

now, vista and win7 are another story. i dont know that they can function properly on a WAN without IPv6, but i dont know cause im not using either atm.

hope that helps more... but as they say around these parts 'your mileage may vary'

JB

I dream of a translated search-able hyperlab... someday. There is a thread on psychonaut on the settings required to mirror the forum but then I realised there wasn't much worth saving from the eventual deletions.

I can't get this to work guys. JBoogie, seeing as you were able to do so, could you kindly provide us with a copy of the site? I know Vesp would be really happy to get one.