|
|
|
|
| Welcome, Guest | Home | Search | Login | Register | |
| Author | Archie (Read 44747 times) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
fogWraith
32 MB  Posts: 59 
|
on: April 20, 2025, 21:26
So I've been hammering away at the keyboard, and there's finally something to show. If anyone actually knows what it is, or has used it before, then I guess it saves me the trouble of explaining an awful lot. The people over at The Serial Port revived Archie a while back, but keeping it alive and functional is another story. Now, I could potentially take the same route and swear like a sailor while setting the official thing up in an emulated environment... y'know, SPARC, Solaris 2.6, port forwarding and the whole shebang, or perhaps even set it up on real hardware with the potential of sudden death and loss of service. Or .. I could write my own software that emulates the real thing, can be run on modern hardware, is cross-platform and might even be a bit more performant. For the latter, time will tell. The first official Beta is online. For web access (no https): http://archie.macdomain.net For Telnet access (username is archie): archie.macdomain.net For Archie clients: archie.macdomain.net Now I haven't had an opportunity to try a whole lot of real Archie clients, but on the Mac side there is Archie and Anarchie (2.0.1 seems to be the last version with Archie support). The server supports both protocol V1 and V5, for some reason V2 through V4 was skipped. During the first run of this particular version, ftp.macintosh.garden is indexed with the mirror username and password, so it's not entirely true to the "anonymous ftp" listings just yet, a new database that is true to the whole anonftp thing is being built offsite. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
cballero
|
1024 MB Posts: 1176 System 7, today and forever
Reply #1 on: April 21, 2025, 01:51
|
Very cool, FW!  I only heard about it way back when, but never used it directly? I was busy doing web searches on the web search giant, AltaVista and Wired Magazine's-slash-Inktomi's very cool (modern url: HotBot) back in my early web searching days (when not preoccupied sucking all the files out of AOL's back-then "huge" shareware servers  and not counting much less-frequent AOL searches and Yahoo search engines with an added directory listing), at least until Google began showing growing strength with their speedy searches Its searches look pretty cool; thanks for building this out! 
|
fogWraith
|
32 MB Posts: 59
Reply #2 on: April 27, 2025, 20:46
|
I've made some progress with the backend, the searching has been somewhat optimized and a rough estimate of searching a bit over 5M records takes less than 100ms to yield results. The crawler has been improved as well, and will be less prone to error out now, even though there is most likely more tweaking required there when it comes to the largest of sites. Database has been cleared out, and a fresh one is slowly being built, right now one can search actual anonymous FTP sites. These include ftp.rfc-editor.org, ftp.aminet.net, ftp.apple.asimov.net, ftp.gnu.org, hobbes.vespernet.net, ftp.belnet.be There's a few more in queue, running them one by one at the moment to see how the latest changes are performing. Had to force dynamic indexing since I did notice that some sites have semi-broken/mangled ls-lR indexes, which messes up the database. Currently queued the following ones for indexing: ftpmirror.your.org, ftp.sunet.se, ftp.lysator.liu.se, ftp.icm.edu.pl, ftp.mirrorservice.org On a sidenote, I previously mentioned that 2.0.1 of Anarchie was probably the last version with Archie support. It wasn't, Anarchie Pro 3.5 seems to be the last version with Archie support.
Last Edit: April 30, 2025, 08:31 by fogWraith
|
cballero
|
1024 MB Posts: 1176 System 7, today and forever
Reply #3 on: April 28, 2025, 23:43
|
Wow, super fast now! And please keep indexing FTP sites they appear to be fading from search engines and Google doesn't list anything ftp-related anymore; and when it used to, it refused to show the ftp directories altogether, which is hugely annoying!  Thanks again! 
|
ClassicHasClass
|
32 MB Posts: 39
Reply #4 on: April 29, 2025, 18:34
|
Excellent work! I've always thought about doing an Archie revival but the indexing was the intimidating part, and you've already jumped into that. Kudos for making it accessible to existing Archie clients, too.
|
fogWraith
|
32 MB Posts: 59
Reply #5 on: April 30, 2025, 06:42
|
The indexing really is the intimidating part, I've had to go back and forth so many times to create something that deals with the majority and obscurity that is FTP servers and the software ... and the database really grows at an alarming rate  Archie revival being one thing (fairly alright process), it is as you say, the indexing that is the killer here, the few sites that are small are far and few in between, all the while the larger ones are either just "large" or monstrous when it comes to the amount of files. As far as the clients goes, I have only had the opportunity to test Anarchie and WSArchie 16- & 32-bit versions, and none of the unix/linux variants, hopefully though these also work. I did notice something that I need to take a closer look at, and that's the handling of very long paths and filenames :/
Last Edit: April 30, 2025, 07:54 by fogWraith
|
Bolkonskij
|
Administrator 1024 MB  Posts: 2023
Reply #6 on: April 30, 2025, 16:07
|
A search-engine for anonymous FTP servers is just such an excellent idea; and one that we've been missing. There are a lot of gems out there on FTPs that nobody knows about. I did a brief search for "macintosh" from my IIci running MacLynx and everything worked wonderfully. I'll dig more into it at the weekend. First impression is good. You do need to know a keyword for what you're search for though. It's hard to explore unless you get creative. If there'd be an alternative option to "browse by category" that'd definitely encourage active exploration but not sure how feasible that would be. Categorizing stuff sounds like an awful lot of work. I somehow still hope we'll eventually find some mirrors of the SUMEX archive (dating back to 1984!!) which eventually morphed into Info-Mac. This was essentially the Macintosh Garden of its day, everything was found+uploaded there. As I understand, they unfortunately were in the habit of "weeding out" old stuff to make space for new things available, resulting in many many many files being lost over the decades. Some stuff has been preserved on floppy / CDs here and there, but my impression is that it was mostly the more popular shareware - but not the hand-crafted b&w artwork, the how-to tutorial text-files, the humorous stuff, the game mods, the little plug-ins, bugfixes etc. etc. Unless I've been overlooking something, I really hope somebody preserved some SUMEX / Info-Mac dumps from various timestamps and can make it available to the public again.
Last Edit: April 30, 2025, 16:15 by Bolkonskij
|
wove
|
1024 MB Posts: 1363
Reply #7 on: April 30, 2025, 16:29
|
I do not know about the InfoMac files, however info mac is still up and I imagine that it is still ran by Dan. I actually just logged into info-mac and I guess I am still a moderator over there. Just came across this from the Internet Archive. I did not dig into it at all, so I do not know what it might actually contain. But here it is. <https://archive.info-mac.org>
Last Edit: April 30, 2025, 16:53 by wove
|
cballero
|
1024 MB Posts: 1176 System 7, today and forever
Reply #8 on: April 30, 2025, 18:33
|
Don't know if any old MUGs uploaed some of their old member's only stuff; I recall certain software only available from there, but I'm almost sure over time, the MG and older Mac Book CDs might have most of such gems covered (hopefully!)
|
Bolkonskij
|
Administrator 1024 MB Posts: 2023
Reply #9 on: May 01, 2025, 07:16
|
Quote from: wove I do not know about the InfoMac files, however info mac is still up and I imagine that it is still ran by Dan. I actually just logged into info-mac and I guess I am still a moderator over there. Thanks, wove! I'm aware of Info-Mac still being kept online by Dan as an archive, ran into it some years ago. (but why oh why only via https ... )I think this is but the latest iteration of Info-Mac, mostly from the Mac OS 8/9/X era. There isn't any of the earlier 1990s or even 80's stuff there. Quote from: cballero Don't know if any old MUGs uploaed some of their old member's only stuff; I recall certain software only available from there, but I'm almost sure over time, the MG and older Mac Book CDs might have most of such gems covered (hopefully!) In terms of most shareware titles, especially games - yes! In terms of the stuff I mentioned above - no! When I was hunting down game mods years ago I found mention of so many being uploaded to Info-Mac but they are nowhere to be found. Like I know somebody in the 1980's did a series of scenarios for Ancient Art of War that I've been looking for. I even know some of those scenario's names! Still, it's really hard to track down these. Same goes for a lot of other stuff, be it niche add-ons for certain programs, or tutorials, or art or else. EDIT: Would it help the project to submit addresses of active anonymous FTP servers? I have a few bookmarks (of mostly Mac related) FTP servers that don't seem to be included yet.
Last Edit: May 01, 2025, 09:39 by Bolkonskij
|
wove
|
1024 MB Posts: 1363
Reply #10 on: May 01, 2025, 14:05
|
My memories of Info Mac origins is very fuzzy. Dan purchased the info-mac domain name. I was not sure whether there was an official InfoMac group to sell it, or he just bought it from squatter. I think he announced it on System7Today and that post might provide more and better information. I am not sure that info-mac ever was a “thing”, but more just a conglomeration that used a shared name. Although I am not positive on this I had thought that info Mac origins go back to a news group, comp.sys.infomac(?) I do remember the info-mac archives were a diverse group of servers with a fair amount of overlap, but each archive did have items and focus that differed from others. I do not recall there ever being any attempt to try and keep the servers synchronized. There was a great deal of stuff to be found in them, much obscure, arcane and just garbage too. I think there was always a sense that much of what info Mac contained should be kept and archived tor the future, but the task of organizing all the stuff was simply beyond the reach of an ad-hoc group, with no real well structured governance.
|
fogWraith
|
32 MB Posts: 59
Reply #11 on: May 01, 2025, 16:34
|
As far as the Archie server and various bits and bobs it provided goes, the Archie project solely handles Archie and nothing more, so don't expect it to be extended beyond what it was The CGI emulation layer is pretty much done, I don't intend to implement the web search feature since I've already built a search engine (which is hosted behind my own DNS). The CGI package is what came with the Archie Beta source code, I believe, if it differs from the "final version"? I'll have to check, and if it does, update my code I might make one more WWW interface available, and that is the ArchiePlex thing that went around, and looks slightly different from what came with the official package. It did have a feature / command for looking up what something is with "whatis" I believe, which is something I might implement at some point, but first comes proper handling of clients and data (and building a database that can be used). It's a shame the latter takes so much time to "get right" and be a universal way of gathering things up whatis In addition to lists of anonymous-ftp-site directories, archie servers maintain a list of program names and a brief description of what they do. This is calledthe whatis database. You can use the "whatis [program-name]" command, and if there is an entry for that program name, the archie server will provide you with the description. Since these descriptions are entered manually by humans, they are sometimes not up-to-date. In addition, occasionally there are old programs in the whatis database which have been removed from ftp sites. As a result, you might occasionally find a description for a program with the whatis command, but not be able to find that program with an archie search. --- Quote from: Bolkonskij Would it help the project to submit addresses of active anonymous FTP servers? I have a few bookmarks (of mostly Mac related) FTP servers that don't seem to be included yet.Yes, it would, I have slowly been picking up FTP sites that are still active, most of them are on the large side of things as to why they have been placed last in the queue so far. It takes a lot of time to index something on the medium - bordering on the large side. But it eventually gets done. Last time I started out with a large site, only indexing a mirrored Wikimedia folder grew the database by over 10 GB alone, I believe. Then again there were files spanning a scary amount of years into the past.
Last Edit: May 01, 2025, 19:17 by fogWraith
|
lauland
|
512 MB Posts: 674 Symtes 7 Mewconer!
Reply #12 on: May 02, 2025, 16:08
|
I'm terribly impressed by everything you've done! Every time I think I've wrapped my head around the scope of your project, you mention something unexpected (at least to me), like CGI support. I've had a lot of fun browsing the ftp sites via your archie. The view is uncluttered and things really stand out, that I've missed by browsing the sites themselves.
|
Bolkonskij
|
Administrator 1024 MB Posts: 2023
Reply #13 on: May 04, 2025, 16:37
|
I'm posting from my IIci, I'll share a few of my Transmit bookmarks. Quick Archie search didn't turn any of them up but forgive me if there's duplicates. ftp.mayn.de/pub/really_old_stuff/apple/ Unix + Apple stuff, German language updaters & more sunsite.icm.edu.pl/pub/mac/ massive file repository, Mac stuff being only a fraction. ftp.funet.fi/pub/ Another big & rather well known repository ftp.ibiblio.org/pub/ Offline right now when I checked - might come back ftp.cc.umanitoba.ca Smaller collection of PC / Mac stuff from the University of Manitoba (?)
|
fogWraith
|
32 MB Posts: 59
Reply #14 on: May 04, 2025, 19:03
|
sunsite is massive as far as i can tell, as to which it is last in the list funet is also stupidly large, they are also amongst the last to be indexed since they mirror EMBL's European Bioinformatics Institute: Big data for the life sciences ... amongst other things I'll have a look at the other ones, cheers
|
|
Pages: [1]
|
| |||||||||||||||
|
© 2021 System7Today.com. |
