Distributed hub system

Ideas for ADC may be presented here for others to review and point out flaws or further improve the idea.
Forum rules
If you have an account on the wiki, remember to update the ADC Proposals page for new ideas.

http://dcbase.org/wiki/ADC_Proposals_list
rhys_rhaven
Newbie
Posts: 9
Joined: 08 Oct 2008, 16:07

Distributed hub system

Post by rhys_rhaven » 08 Oct 2008, 16:16

So the wiki on adc.sourceforge.net doesn't work. There seems to be pretty sparse info on ADC save for the protocol page. Such as the ADCS protocol. This is all rather frustrating. Anyway, my idea that I'm doing research into is a distributed hub system. Very much how seperate IRC servers work together as a single network.

I assume this has been done. I want to encrypt it by default, and also the big thing I want among this network is a distributed list of clients. If one server goes down, the other servers know about it, send out redirects to the crashed server's clients. Still client/server architechture, but a hellva lot more redundancy.

I want to make this a proper extension. I've been lookng into distributed data structures to handle the shared system. What do people think?

Rhys

Toast

Re: Idea for Extension

Post by Toast » 08 Oct 2008, 17:28

Sounds like an interesting extension if it can be done, as for the wiki for ADC ive been talking to DCDev about this and hope it will get resolved soon for all Devs/Users sake.

Pietry
Senior Member
Posts: 328
Joined: 04 Dec 2007, 07:25
Location: Bucharest
Contact:

Re: Idea for Extension

Post by Pietry » 08 Oct 2008, 18:25

adc wiki should work if you get registered. it got spammed and non registered modification was restricted as a security measure...
Just someone

Pietry
Senior Member
Posts: 328
Joined: 04 Dec 2007, 07:25
Location: Bucharest
Contact:

Re: Idea for Extension

Post by Pietry » 08 Oct 2008, 18:42

And as for the extension, sounds interesting, and it could lead to a more decentralized ADC network. Sounds familiar to bigmuscle's dcn draft http://www.adcportal.com/forums/viewtop ... f=10&t=253
Just someone

rhys_rhaven
Newbie
Posts: 9
Joined: 08 Oct 2008, 16:07

Re: Idea for Extension

Post by rhys_rhaven » 08 Oct 2008, 18:52

Main difference is this is NOT to be decentralized. My work deals with private networks that have high bandwidth, intranets. Such as Unversities etc etc. Decentralization is inefficient for those kind of networks. This is a matter of availability between servers, not between clients. This should require no change to clients, save for maybe in session redirects.

Toast

Re: Idea for Extension

Post by Toast » 08 Oct 2008, 19:57

Looking forward to reading the draft whenever your planning to publish that :)

blastbeat
Member
Posts: 53
Joined: 10 Jan 2008, 19:56
Contact:

Re: Idea for Extension

Post by blastbeat » 08 Oct 2008, 20:21

i tried some time ago the same direction and called it "LINK"
the main problem i think are sids. my proposal would be that every hub in the network has a certain sid namespace, so that there is no need to convert sid to cid etc. with this method a hub can dispatch messages without altering it

rhys_rhaven
Newbie
Posts: 9
Joined: 08 Oct 2008, 16:07

Re: Idea for Extension

Post by rhys_rhaven » 08 Oct 2008, 20:40

The reason I haven't written anything final for it is because I don't know how to make it into a "protocol" as it were. I have tons of notes written down about behavior, data structures and the like. All of my stuff is for my implementation of it though, which involves 100% availability, not simply linking.

For things to be available in case of crash, they each have to have at minimum the addresses of all the clients, and from that if a server goes down its information can be rebuilt by querying the clients. I assume a new SID would need to be assigned as well.

Also, I have very little info about ADC servers in general. No wiki, all I have is the protocol and my notes that condense the protocol. Thus I have no idea how things are generally done. Search for example. To perform the search, the hub keeps a copy of all filelists? (I assume in a few big hashtables, based on filename and TTH) And when the hub sends the result, it identifies file owners by SID? All of this I don't know, and I'm kinda making it up as I go. Which is bad.

blastbeat
Member
Posts: 53
Joined: 10 Jan 2008, 19:56
Contact:

Re: Idea for Extension

Post by blastbeat » 08 Oct 2008, 21:26

all client identification in a hub is done by sid. when you connect 2 hubs, you have maybe 2 users with same sid. so you have to translate every sid between hubs or you divide the complete sid namespace.

Pietry
Senior Member
Posts: 328
Joined: 04 Dec 2007, 07:25
Location: Bucharest
Contact:

Re: Idea for Extension

Post by Pietry » 09 Oct 2008, 07:32

Searches are done only in the clients. The hub only acts as a relay. When a client wants to search, it sends a search message to the hub which broadcasts the message to all connected clients. Each client then looks into its own file list and if the case, returns results directly to the searching client via UDP if active mode, or via the hub if passive mode.
Adcportal is always a good place to ask your questions about the protocol, and since there are a lot of developers here don't hesitate to ask you will surely get your answer. My sole recommendation is to ask every adc protocol question in a separate thread ( might be helpful to someone else too, mixing them all up in one thread is hard to follow )
Just someone

Locked