Author Topic: Righto - We need a PROPER library  (Read 180 times)

no1uno

  • Global Moderator
  • Foundress Queen
  • *****
  • Posts: 681
Righto - We need a PROPER library
« on: October 18, 2009, 09:34:10 AM »
There are a shitload of journal articles that are already on the web, on our harddrives and on online file servers (not to mention the ones we've all lost over time)...

I know Sedit has a library on here and numerous people have ftp accts... I and several others have decent collections online (as well as elsewhere)

But it don't work - we have rhodium mirrors (plus rhodium lookalikes), old pdf archives of the same, etc. plus things like this (which is fucking extensive), but it isn't all available or even indexed appropriately

What we need is something people can look through, browse through, by type of chemistry, then by group of compounds, then by individual compound type, look at individual files, check the abstract and then decide whether or not to download them. Even if we simply connected a library up to the existing sources online, then that would be fucking huge (SM and others have massive amounts of pdf files hosted, so does erowid, Japanese, Brazillian, Chinese, etc. chemistry journals, even the Rhodium pdf indices)...

What we would need is a url - somewhere we can put the library, then set it up with a php/mysql server with admins and editors, and then probably set it up so that the various staff could add links, the abstract, the journal name, doi, authors to a form, choose which subject, category, subcategory and specific type of compound to which it relates and BOOM, the various page(s) update to reflect that new information and that article is from then on, accessible to all searchers for truth.

From then on, whenever the admins/editors see a file uploaded somewhere, they simply grab the abstract, etc. and fill in the form page on the library acct.

"...     "A little learning is a dang'rous thing;
    Drink deep, or taste not the Pierian spring:
    There shallow draughts intoxicate the brain,
    And drinking largely sobers us again.
..."

Vesp

  • Administrator
  • Foundress Queen
  • *****
  • Posts: 3,130
Re: Righto - We need a PROPER library
« Reply #1 on: October 18, 2009, 09:51:18 PM »
What program would even work for that?
Considering a lot of them are science journals - might it be wise to make it a "visible by tor" only sort of site?
Bitcoin address: 1FVrHdXJBr6Z9uhtiQKy4g7c7yHtGKjyLy

no1uno

  • Global Moderator
  • Foundress Queen
  • *****
  • Posts: 681
Re: Righto - We need a PROPER library
« Reply #2 on: October 19, 2009, 08:13:33 PM »
Nope, I won't touch tor/privoxy - anything developed by and for the US Navy, can be read by them - quite simply the biggest diameter pipes for that first communication between your PC and the network, is 99% certain to be owned & operated by the DoD. Nobody else has the time, resources or interest to develop it, so what does that mean? It means that first communication, the one with YOUR IP/MAC addy, can and quite probably is, traced. Why don't we hear this? Cos the jihadists and other fuckups think tor hides them from the DoD, so do the rockspiders...
"...     "A little learning is a dang'rous thing;
    Drink deep, or taste not the Pierian spring:
    There shallow draughts intoxicate the brain,
    And drinking largely sobers us again.
..."

Vesp

  • Administrator
  • Foundress Queen
  • *****
  • Posts: 3,130
Re: Righto - We need a PROPER library
« Reply #3 on: February 23, 2010, 04:36:01 AM »
I'm willing to work on this, I would just need help with the MySQL, PHP, etc set up, -- But when it comes to adding files, I'd be happy to help add the files, abstracts, find them, and so on.

What is the best strategy to go about doing this?
I assume download all of erowids, and gather everyones who is willing to share and work on putting them in the correct structure for abstract & file is for creating the library to get it working properly.  So, I guess in order to know how to set up the files, we'll need to set up the MySQL, etc first. Do you have any idea as how to properly start organizing the journals, etc? I've thought about it, but it seems more complex then expected.
« Last Edit: February 23, 2010, 05:06:29 AM by Vesp »
Bitcoin address: 1FVrHdXJBr6Z9uhtiQKy4g7c7yHtGKjyLy

timecube

  • Subordinate Wasp
  • ***
  • Posts: 230
Re: Righto - We need a PROPER library
« Reply #4 on: February 23, 2010, 06:04:44 AM »
Do the files uploaded with forum posts all get deposited in the same place?
It would be nice if they could all be browsable from one place.

Something I've noticed is the tendency to post external links to documents rather than uploading them directly to the site, and while it isn't practical to archive every resource, I know it's frustrating when browsing the Hive archives and coming across links to websites that don't exist anymore.

Just some thoughts.

Vesp

  • Administrator
  • Foundress Queen
  • *****
  • Posts: 3,130
Re: Righto - We need a PROPER library
« Reply #5 on: February 23, 2010, 06:25:10 AM »
Yeah, that is annoying -- but one also must consider the stupid copyright laws one might have problems with if every resource is uploaded.... kind of like we've been doing in a lot of places here (thinking about making ALL the FTP's password only, but the same password for each one so it can't be indexed by Google, or viewed via links on other sites...  but still accessible easily to the vespiary and other people at SM, BL, WD, etc.

Hopefully soon I can locate a server or two for real cheap and set it up to have some impressive abilities such as unlimited MySQL databases, self signed SSL/HTTPS certificates, dedicated IP, and unlimited FTP accounts.
I am not sure if it will be more reliable in up time --pretty sure it won't be when it comes to that, but I think it will offer a lot of benefits such as total control over the site, etc -- might have more security issues, however?

Either way, I think that will significantly help with the ability to store, share, archive, etc useful resources and make this much needed library-- assuming that the servers do not get maxed out. I am not sure how one determines if a server will be overloaded or not?

Either way, I think I can get a decent server for free, and another one that is probably better for ~60 bucks. This ought to be enough for a while.
Bitcoin address: 1FVrHdXJBr6Z9uhtiQKy4g7c7yHtGKjyLy

t8er

  • Larvae
  • *
  • Posts: 25
Re: Righto - We need a PROPER library
« Reply #6 on: February 23, 2010, 07:30:23 AM »
t8er knows a bit of php and mysql :) and willing to help  ;D
Fizz, PoP, BANG!

shroomedalice

  • Guest
Re: Righto - We need a PROPER library
« Reply #7 on: February 23, 2010, 11:17:38 AM »
shroom is a c coder but that doesnt change the fact that the easier it is to find the less the LE has to do to prove that is worth banning.

I belive the group should stick close together and everything should be scattered. this I belive is our only hope of servival.

if we have a central repository then take a look at rhodium or rogue. ye rogue came back up but they hacked the fuck out of him in a second.

best we make rings of people that have been together for a long time and then slowly induct those we trust. this must not die out becouse
we want hits on our sites.

I will translate the stuff and post it here when I have time but as I have said before some things must stay hidden.

as for mr wizzard I have the up most respect for him and think that should he want to know anything at all he is more than worthy.

be carefull of fester though he will tell the world and fuck it all.

Enkidu

  • Global Moderator
  • Foundress Queen
  • *****
  • Posts: 677
Re: Righto - We need a PROPER library
« Reply #8 on: February 23, 2010, 07:11:59 PM »
if we have a central repository then take a look at rhodium or rogue. ye rogue came back up but they hacked the fuck out of him in a second.

Sorry to pull the thread off topic... I don't remember RS getting hacked in recent history, unless you're talking about a figurative hacking. IIRC we were hacked after the iDefense thing, but that was ancient history, and the site was around long after that.

I do think that the library would be best password protected and the pass given to only posting/contributing members, and everyone who has access should be required to submit information.

Vesp

  • Administrator
  • Foundress Queen
  • *****
  • Posts: 3,130
Re: Righto - We need a PROPER library
« Reply #9 on: February 23, 2010, 11:28:52 PM »
Well, this wouldn't be "Open" to anyone -- but perhaps only 5-10 different people, who then can post it on other sites and share the information. Also having various peoples download it -- so if something ever does happen to the main library, it can be brought back easily. And these 5-10 people would be well known members who are trusted with in our group of like minded people.
Bitcoin address: 1FVrHdXJBr6Z9uhtiQKy4g7c7yHtGKjyLy

Enkidu

  • Global Moderator
  • Foundress Queen
  • *****
  • Posts: 677
Re: Righto - We need a PROPER library
« Reply #10 on: February 23, 2010, 11:52:56 PM »
Well, this wouldn't be "Open" to anyone -- but perhaps only 5-10 different people, who then can post it on other sites and share the information. Also having various peoples download it -- so if something ever does happen to the main library, it can be brought back easily. And these 5-10 people would be well known members who are trusted with in our group of like minded people.


shroomedalice and I were suggesting that only select people would be able to read/download, let alone edit. It all goes back to keeping the site somewhat on the down low. Probably the most important thing to strive for in a site like this a low signal to noise ratio, which can be accomplished either by strict moderation (which doesn't seem to be desired) or growing slowly.

Vesp

  • Administrator
  • Foundress Queen
  • *****
  • Posts: 3,130
Re: Righto - We need a PROPER library
« Reply #11 on: February 24, 2010, 12:00:40 AM »
Yeah, I am not interested in quantity, but advancement.

Quote
Probably the most important thing to strive for in a site like this a low signal to noise ratio, which can be accomplished either by strict moderation (which doesn't seem to be desired) or growing slowly.

I see strict moderation as a last resort sort of thing.  I think we all have a common goal, and will work together to keep things clean, clear, novel, interesting or at least useful.
Bitcoin address: 1FVrHdXJBr6Z9uhtiQKy4g7c7yHtGKjyLy

Naf1

  • Foundress Queen
  • *****
  • Posts: 753
Re: Righto - We need a PROPER library
« Reply #12 on: February 24, 2010, 01:01:52 AM »
Well if anyone needs somewhere to start I have uploaded a couple of thousand pdfs and djvu files that were part of the Rhodium archive, they can be accessed and indexed via these links;

PDF
http://127.0.0.1/rhodium/Rhodium/pdf/

DJVU
http://127.0.0.1/rhodium/Rhodium/djvu/

The thing that makes ScienceDirect, RSC etc etc so good, is that fact that someone has typed out the abstract and title and author and all the relevant information for every paper they have. That information is then entered into a searchable database, so you can search for keywords also (like google). I guess we could just look up the paper in question at one of those sites and copy and paste the information (if the pdf does not allow seletion of text).

Vesp

  • Administrator
  • Foundress Queen
  • *****
  • Posts: 3,130
Re: Righto - We need a PROPER library
« Reply #13 on: February 24, 2010, 01:18:12 AM »
Thank you.

How about this software to start setting it up?

http://pkp.sfu.ca/?q=ojs

Quote
"Scholars need the means to launch a new generation of journals committed to open access, and to help existing journals that elect to make the transition to open access..."
Budapest Open Access Initiative, 2002

Open Journal Systems (OJS) is a journal management and publishing system that has been developed by the Public Knowledge Project through its federally funded efforts to expand and improve access to research.


Looks like it is essentially a program that would be similiar to the one science direct would use.

Here are some others that are listed and could prove to be better/more useful then the one above: http://oad.simmons.edu/oadwiki/Free_and_open-source_journal_management_software


What one do you think would be the best?
« Last Edit: February 24, 2010, 01:20:05 AM by Vesp »
Bitcoin address: 1FVrHdXJBr6Z9uhtiQKy4g7c7yHtGKjyLy

Naf1

  • Foundress Queen
  • *****
  • Posts: 753
Re: Righto - We need a PROPER library
« Reply #14 on: February 24, 2010, 01:36:32 AM »
Looks excellent! Maybe you should trail it?

Vesp

  • Administrator
  • Foundress Queen
  • *****
  • Posts: 3,130
Re: Righto - We need a PROPER library
« Reply #15 on: February 24, 2010, 01:38:27 AM »
Planing on it, when I get the time and figure out exactly what should be done.
This also wouldn't be a bad source to get everything started with...
http://pdf-search-engine.com/

+ I've heard if you have a socialist IP address, you can get a ton of free journals, so I'll have to check into that-- but I guess the existing ones that we've already got ought to be worried about most.
Bitcoin address: 1FVrHdXJBr6Z9uhtiQKy4g7c7yHtGKjyLy

Naf1

  • Foundress Queen
  • *****
  • Posts: 753
Re: Righto - We need a PROPER library
« Reply #16 on: February 24, 2010, 02:10:27 AM »
I downloaded that program (open journal system)to assess the feasibility, it appears you would upload everything to the required directory (like the forum) then there is an installation page you start at. If you need assistance dont hesitate I would be very willing to help with such a great idea, no1uno may want to handle this part? as its his baby? If you want we could just test it by uploading it to someones ftp then testing and ironing out the bugs before it gets put up officially?

Vesp

  • Administrator
  • Foundress Queen
  • *****
  • Posts: 3,130
Re: Righto - We need a PROPER library
« Reply #17 on: February 24, 2010, 02:15:51 AM »
Quote
If you need assistance dont hesitate I would be very willing to help with such a great idea, no1uno may want to handle this part? as its his baby?

Yeah, I really would like to hear no1uno's comments on this before really anything happens, hence my hesitation. Does it look really labor intensive per file? or is it pretty easy to deal with everything?
Bitcoin address: 1FVrHdXJBr6Z9uhtiQKy4g7c7yHtGKjyLy

Naf1

  • Foundress Queen
  • *****
  • Posts: 753
Re: Righto - We need a PROPER library
« Reply #18 on: February 24, 2010, 02:21:54 AM »
I did not see the actual interface, it was relatively small download containing alot of php, html, xml files etc. One would assume these would be uploaded to the ftp then you would surf to the installation page via html. You would imagine it will be like most other automated DB programs, being the fields will already be set and you would just have to type title, numbers, and abstract etc, but thats purely speculation.

Vesp

  • Administrator
  • Foundress Queen
  • *****
  • Posts: 3,130
Re: Righto - We need a PROPER library
« Reply #19 on: February 24, 2010, 02:24:34 AM »
Yeah, and probably a MySQL DB would have to be added and connected.. This shouldn't be to hard at all, hopefully. Just very very time consuming.
Bitcoin address: 1FVrHdXJBr6Z9uhtiQKy4g7c7yHtGKjyLy