AUT0 - Network connection auto detection extension

Ideas for ADC may be presented here for others to review and point out flaws or further improve the idea.
Forum rules
If you have an account on the wiki, remember to update the ADC Proposals page for new ideas.

http://dcbase.org/wiki/ADC_Proposals_list
Locked
Toast

AUT0 - Network connection auto detection extension

Post by Toast » 13 Jan 2011, 18:09

the 'AUT0' extension (network connection auto detection extension).
Rationale: Detect if client is capable of initiating active connections.

After logging in:

Client -> 'HCHK 12345 ABCD'.

Server sends a UDP packet containing token to the client's IP address at the given port as shown in the INF message.

Server -> 'ICHK ABCD'

The server should send it from a UDP socket bound to the same port as the TCP server, which means the server will
have to listen to both TCP and UDP.

If client receives the packet, it knows it can receive UDP connections, and will advertise it in the INF message as
a supported feature.

If the client does not receive any UDP packets within a few seconds, it MAY try to reconfigure the router using
UPnP/ZeroConf, and issue a HCHK command to the server again.

If the client still doesn't receive any UDP packets, hole punching can be tried.
The server will send a UDP packet to the hub (using the port of the TCP server), and reissue the HCHK command.
The UDP packet SHOULD be echoed by the hub.
This UDP packet should contain simply 'HECH {cid} {token}' (Hub echo).

The hub should send a packet containing the token back:
'IECH {token} {host:port}', aswell as the same message via TCP.

If the client receives the message via UDP, it should now be able to determine the type of NAT.
If the client receives the message via TCP only it knows it has a firewall blocking icomming communication.
If the client does not receive the message, it should assume a firewall is blocking all UDP communication,
and resume in passive mode.

Requirements:
'AUT0' in the extensions message (SUP) for client and hub.
Server will also listen to UDP messages on the same port as the TCP port.

The server MUST now respond to the 'HCHK' command via TCP
The server MUST now respond to the 'HECH' command via UDP.

The client will always initiate communication.

-------------------------------------------------------------------------------

Syntax: HCHK {port} {token}
- port is a 16-bit port number where the client is listening for packets.
- token is 4 bytes base32 encoded data specified by the client.

Example:
Client: 'HCHK 1511 BACD' (tcp)
Server: 'ICHK BACD' (udp)

-------------------------------------------------------------------------------

Syntax: HECH {cid} {token}
- cid is the client ID.
- token is 4 bytes base32 encoded data specified by the client.

Example:
Client: 'HECH 3NGFVJUDBRHX2CYRJGQ5HACRDM5CTNLGZ36M64I BACD' (udp)
Server: 'IECH BACD 192.168.0.1:1512' (udp)
Server: 'IECH BACD 192.168.0.1:1512' (tcp)

Security considerations:
The hub must verify that IP address where the HECH command originated from
matches the IP address where the user with the given CID is connected from.
If the CID and IP does not match, or the CID is not used the hub MUST ignore
the request.

Pirre
Junior Member
Posts: 31
Joined: 10 Nov 2009, 11:57

Re: AUT0 - Network connection auto detection extension

Post by Pirre » 14 Jan 2011, 07:43

It would been a nice thing to have and surely needed.

It will need also the client sending both his TCP and TLS port in his INF as this is not the case now.

And there is 1 thing using UDP will not let you verify that there is a real clients TCP/TLS port listning on the clients side as far i know and you need a extra listner on the servers side.

Why not having the hub send a real tcp (with or without TLS layer depending on the SUP info form the client) from a diff port then the hubs port and realy negotiate like the first steps in the client-client file transfer using the RCM (one with ADC0 feature in string if client supports and one without), that way you do not need the client ports in the INF (you will get them from the clients CTM).

Using this would be a real test that it will be a TCP packet that is allowed (because a UDP is reaching the port does not mean a TCP can) and you can be sure its the DC client who is listning on that port and not something else.

I have been on a P2P before where they had a UDP system like proposed here and the client used port 443 default. That test failed completely the day Skype was launched and used same default port. And yes i know the client will verify that the port is in use but the above TCP verification avoids such probs completely and avoid sending extra INF fields and avoids a extra listning socket hubs side.

My 2 cents :lol:

Pretorian
Site Admin
Posts: 214
Joined: 21 Jul 2009, 10:21

Re: AUT0 - Network connection auto detection extension

Post by Pretorian » 21 Apr 2011, 14:14

In Janurary;
[2011-01-16 01:03:32] <Pretorian> On a different note, what does AUT0 do? I don't get it. It's... emulating RCM/CTM?
[2011-01-16 01:06:39] <Pretorian> The port in CHK is useless information (it's already supplied in INF) and the CID in ECH is also pointless (it's an H-type message, so the hub already knows the CID)...
[2011-01-16 01:09:08] <Pretorian> I'm not entirely sure it's even needed. Clients can a) assume active b) fallback to NAT-T if connection fails for X tries.
In February;
[2011-02-25 21:57:04] <Pirre> did you ever had a look @ that AUTO extention proposal ? thats infact what is needed is 1 form or another but think it needs "sponsors" lol
[2011-02-25 21:57:22] <Pirre> is=in
[2011-02-25 21:57:55] <Pretorian> IIRC, it's pointless with NAT-T?
[2011-02-25 21:58:04] <Pretorian> Or what was it about...
[2011-02-25 21:58:38] <Pirre> hub detects if client is active or passive and informs client
[2011-02-25 21:59:08] <Pretorian> Implement NAT-T and don't care?
[2011-02-25 21:59:52] <Pirre> and no its not useless because nat exist as 1 if people put themself active nat is not used and 2 not all routers support nat
[2011-02-25 22:01:45] <Pirre> these "misconfigurations" cause fustration for the client and endless ctm strings in the hub
[2011-02-25 22:02:10] <Pretorian> 1) If users put themselves as "active" and the client detect that the client can't connect (for some reason), the client could default back to NAT-T...
2) True, not all routers support NAT. However, I wonder whether people will understand "X hub tells you to check your connectivity" vs the client telling them so (or simply asking in the hub).
[2011-02-25 22:02:29] <Pretorian> But if there's an implementation, I'll of course add it to ADC-Ext...
[2011-02-25 22:03:24] <Pirre> there is only a proposision made by toast, a copy of another old spec ...
[2011-02-25 22:03:40] <Pretorian> I'd have to re-read the proposal and change the writing to a simpler format since it's slightly confusing, I think, as it is now.
[2011-02-25 22:03:52] <Pirre> a bit yes :)
[2011-02-25 22:05:08] <Pretorian> Also, since I'm a little sceptical myself about its usefulness, I'd have to convince myself of its usefulness to include (or not) a "you should implement this regardless of NAT-T" comment...
[2011-02-25 22:13:23] <Pretorian> Simple C-C for TCP could be done without an extension today...
[2011-02-25 22:13:55] <Pirre> true but then the client cant be "informed"
[2011-02-25 22:14:07] <Pretorian> STA or MSG...
[2011-02-25 22:14:13] <Pirre> infact inhere is a very basic tester
[2011-02-25 22:15:55] <Pirre> then the client needs to send "something" to inform the hub he wants a test and can/will/use/receive the result
[2011-02-25 22:16:06] <Pretorian> Technically, UDP could be tested also... E.g., assume that the hub "searches" for an item it knows/assumes the client has in its share...
[2011-02-25 22:16:59] <Pirre> yep there a same system as the "testsurs" could be used
[2011-02-25 22:17:20] <Pirre> a fake file that give a sch result
[2011-02-25 22:17:25] <Pretorian> Right.
[2011-02-25 22:17:54] <Pretorian> Doesn't even have to be a fake file...
[2011-02-25 22:17:59] <Pretorian> Simply search for "a".
[2011-02-25 22:18:27] <Pirre> lol
[2011-02-25 22:18:46] <Pirre> if a user has shares yes lol
[2011-02-25 22:19:18] <Toast> HTC Desire rox :D
[2011-02-25 22:19:26] <Pretorian> Well, highly likely.
[2011-02-25 22:20:09] <Pirre> hehe new toys :)
[2011-02-25 22:20:58] <Pretorian> Hm, it should be the other way around, shouldn't it? Client searches for something... Not the hub.
[2011-02-25 22:21:24] <Pirre> no a BSCH should reach him
[2011-02-25 22:21:49] <Pretorian> Well, URES is what you want to test...
[2011-02-25 22:22:18] <Pirre> thats outgoig no ?
[2011-02-25 22:22:34] <Pretorian> Incoming from the client's point of view.
[2011-02-25 22:23:15] <Pretorian> Client searches and will get a URES, which is what you want to check; whether the client can receive UDP traffic.
[2011-02-25 22:23:49] <Pirre> you 100% right
[2011-02-25 22:24:02] <Pretorian> Likewise with CTM... Client tries to connect to the hub...
[2011-02-25 22:24:11] <Pretorian> (or, rather, some pseudo client.)
[2011-02-25 22:24:54] <Pretorian> (Doesn't really matter.)
[2011-02-25 22:25:24] <Pretorian> (Well, most hubs probably don't understand HCTM...)
[2011-02-25 22:26:01] <Pirre> infact the hub should be "passive client" emulating in both cases imho
[2011-02-25 22:26:53] <Pirre> but not for the searches offcourse there you are correct
[2011-02-25 22:27:35] <Pretorian> So just say "if client and hub support AUTH, the client will perform a HCTM [and can simply disconnect if the hub connects] and a HSCH [searching for FOOBAR or something specifically stated]".
[2011-02-25 22:29:48] <Pirre> true with one thing , hub should start with a rcm to receive the port and 2 of them 1 with feature ADCS and one without
[2011-02-25 22:30:12] <Pirre> as far i know the client does not send port info ... in INF
[2011-02-25 22:30:44] <Pretorian> The hub doesn't have to start with RCM if the client does a CTM on connect...
[2011-02-25 22:30:49] <Pretorian> It does.
[2011-02-25 22:31:03] <Pretorian> Remember that an RCM generates a CTM.
[2011-02-25 22:31:39] <Pretorian> So you place the responsibility with the client handling the message of notification.
[2011-02-25 22:31:59] <Pirre> yes
[2011-02-25 22:32:07] <Pretorian> (Say, if the client hasn't received a connection/RES after 30 seconds, it can inform the user.)
[2011-02-25 22:32:15] <egik> answer for dcpp team: where in core may catch sending RES adccommand as sample string or cast in string for adding this strings in cmd debug (with AdcDebug equal 1 this strings don't shown) or this is unreal?
[2011-02-25 22:32:21] <Pretorian> But yes, you want to have a FCC (but you aren't required).
[2011-02-25 22:33:07] <Pretorian> (By 'have to' I mean 'unless you want to generate seamingly meaningless traffic for hubs'.)
[2011-02-25 22:34:13] <Pirre> thats exact what needs to be discused :)
[2011-02-25 22:36:06] <Pretorian> egik: I can't answer you, but you can always (I say 'always', even if I don't maintain the question tracker) ask in Launchpad or on ADCPortal (if you don't get an answer here).
[2011-02-25 22:38:59] <Pretorian> Simplest SCH is probably searching for the TTH of an empty file.
[2011-02-25 22:39:29] <Pirre> that will not give a res
[2011-02-25 22:39:38] <Pretorian> Why not?
[2011-02-25 22:40:46] <Pirre> client will only send res if tth found ...
[2011-02-25 22:41:03] <Pretorian> The hub responds, remember.
[2011-02-25 22:41:13] <Pirre> grrrrrrrrr lol
[2011-02-25 22:41:26] <Pirre> true
[2011-02-25 22:41:39] <Pretorian> So hub can always respond to the empty file TTH.
[2011-02-25 22:41:44] <Pirre> yep
[2011-02-25 22:41:54] <Pretorian> (or atleast in a HSCH context)
[2011-02-25 22:42:53] <Pirre> should solve udp
[2011-02-25 22:43:54] <Pretorian> And client simply will do a HCTM and disconnect once the token matches and a connection is established.
[2011-02-25 22:44:40] <Pirre> also true
[2011-02-25 22:45:16] <Pirre> 2 times if hub supports adcs verification and that will be a prob
[2011-02-25 22:45:25] <Pretorian> Right.
[2011-02-25 22:45:31] <Pirre> a normall adc has no certs
[2011-02-25 22:46:37] <Pirre> same will be the case for udps if its going to use diff ports
[2011-02-25 22:50:10] <Pretorian> I still wonder whether the client telling the user "it seems you don't support incoming connections, please do X" is useful. I mean, the user will notice quite quickly... But sure, if you want to get rid of annoying users. :)
[2011-02-25 22:50:42] <Pretorian> (Granted, the user can't tell for sure whether searches work immediately, unless they search for something that should generate responses.)
[2011-02-25 22:52:25] <Pirre> problem is that the "source" telling the client its settings are wrong should be more or less relaible, otherwise you could indeed have a "failiure" counter
[2011-02-25 22:52:38] <Pretorian> Right.
[2011-02-25 22:54:27] <Pirre> 2300 CTM and RCM's from 1 client that is misconfigured
[2011-02-25 22:54:31] <Pirre> lol
[2011-02-25 22:54:39] <Pretorian> o_O
[2011-02-25 22:55:02] <Pirre> big shares and udp works on him
[2011-02-25 22:55:12] <Pirre> but not tcp :)
[2011-02-25 22:58:14] <Pirre> time for a :beer: intresting convo
[2011-02-25 23:00:27] <Pirre> what you sugest can even be done scriptwise
[2011-02-25 23:00:59] <Pretorian> :)
[2011-02-25 23:01:03] <Pirre> so if you "word" it i will try to "make" it lol
[2011-02-26 00:14:01] <Pirre> only see 1 prob with your solution regarding auto pret and that it uses the server port to do the "verif" have no clue how well arnes shared_pointers deal with it ... think we need him or collogic to answer that unless you know more
[2011-02-26 00:45:05] <Pretorian> Pirre: Implementation issue...
[2011-02-26 00:45:52] <Pretorian> Just connect to the port...Really simply...
cologic;
[2011-02-02 11:01:28] <cologic> Oh, 'AUT0'. Cute pun.
[2011-02-02 11:02:22] <cologic> "The server should send it from a UDP socket bound to the same port as the TCP server, which means the server will
have to listen to both TCP and UDP." seems to address that concern though.
[2011-02-02 11:02:47] <cologic> I'm not sure Toast quite understands UDP though, since it's not technically entirely coherent
[2011-02-02 11:05:00] <Pirre> as i understoud it (and maybe i am wrong) , the verification uses a UDP packet to the client listners port what is TCP for xf
[2011-02-02 11:05:34] <cologic>
The hub should send a packet containing the token back:
'IECH {token} {host:port}', aswell as the same message via TCP.

If the client receives the message via UDP, it should now be able to determine the type of NAT.
If the client receives the message via TCP only it knows it has a firewall blocking icomming communication.
If the client does not receive the message, it should assume a firewall is blocking all UDP communication, and resume in passive mode.
[2011-02-02 11:05:45] <cologic> So, it seems to probe each protocol separately.
[2011-02-02 11:06:34] <cologic> For the client, at least.
[2011-02-02 11:07:35] <cologic> But I also don't understand the separation between 'hub' and 'server'. They're different entities in that description, but what other machine is supposed to play the role of server?
[2011-02-02 11:07:49] <cologic> Since it uses I/H commands
[2011-02-02 11:09:01] <Pirre> for me the logic is that when verifying the clients connection the client is the server
[2011-02-02 11:09:19] <Pirre> as thats exactly what is tested
[2011-02-02 11:10:03] <cologic> so why have the 'client' as a separate role?
[2011-02-02 11:10:24] <Pirre> have no clue there
[2011-02-02 11:11:35] <cologic> one's interpretation of that protocol depends on how one assigns 'client' and 'server', so until/unless that's clarified I think dealing with that proposal involves chasing ghosts
[2011-02-02 11:12:51] <Pirre> ye but a pitty it drags on that as its a nice feature capable of eliminating a lot of unneeded traffic in a hub
[2011-02-02 11:12:53] <cologic> it is possible to imagine a protocol where one only has two machines but the roles are factored such that there's a 'floating' role worth splitting out, but I don't think (?) that's what Toast is trying to communicate
[2011-02-02 11:12:56] <cologic> Yes
[2011-02-02 11:13:15] <cologic> So, try to get Toast to clarify it I guess - it looks like you already have, without success, though
[2011-02-02 11:13:33] <Pirre> not that good with words lol
[2011-02-02 11:14:02] <cologic> (oh, other WINE quirk - ctrl-left/right in multiline edit controls just kind of stop at line boundaries. Other cursor movement keys mostly work though.)
[2011-02-02 11:15:10] <cologic> Well, you're partly objecting on other bases, too - e.g. the Skype/port 443 concern.
[2011-02-02 11:15:20] <cologic> Have you asked him in this hub about it?
[2011-02-02 11:15:49] <Pirre> not yet, but i think the concept is taken from another hubsoft
[2011-02-02 11:16:14] <cologic> Given his response to FleetCommand's criticisms of his English, he's got no basis to ignore you just because you don't write even somewhat less smooth English. :p
[2011-02-02 11:16:41] <Pirre> o toast will not do that :)
[2011-02-02 11:17:22] <cologic> As in, Toast is just describing what some existing hub software already does?
[2011-02-02 11:17:48] <cologic> Seems pointless without client support.
[2011-02-02 11:20:30] <Pirre> ye thats what i mean, think it was uhub but not 100% sure and there are already some clients (maybe svn) that have it implenetd
[2011-02-02 11:21:45] <cologic> Oh. Then it's even more important that either someone provides a more readable description of what they're doing (if it's a communication problem via Toast) or that some of the weirdness might be fixed (if Toast is describing it correctly).
[2011-02-02 11:22:29] <cologic> And also means that Toast (or someone familiar with that hub software - e.g. jvk) is uniquely qualified to elaborate.
[2011-02-02 11:23:06] <Pirre> will ask around :)
[2011-02-02 11:24:22] <cologic> I broadly agree though that this is an obviously useful extension and is worth getting right.
[2011-02-02 11:27:57] <Pirre> https://github.com/janvidar/uhub/blob/m ... nsions.txt only ref i can find and that it is used ...
[2011-02-02 11:28:27] <cologic> Oh, looks like Toast copy/pasted from that

Pretorian
Site Admin
Posts: 214
Joined: 21 Jul 2009, 10:21

Re: AUT0 - Network connection auto detection extension

Post by Pretorian » 10 Mar 2013, 14:46

Had a new discussion today about this, and I had roughly the same ideas as my previous noting in the paste above, but here is a more constructed way.
NATO - Network connection auto detection extension

The client have no way to verify whether it can manage incoming TCP and/or UDP connections and messages. This extension intend to provide this capability by making the hub responsible for helping the client ascertain this information.

Signal AUTO in the SUP to indicate support for this.

The client may send a HCTM where the hub connects to the client as in a normal client-client connection. The client may disconnect after it has received the INF (matching the token only) in the client-client connection. If no connection is established within two minutes (120 seconds), it means the client has probably failed to properly set up the network for active mode.

The client may send a HSCH for the 0-byte file hash where the hub responds with a URES for that file. Note that the hub must a) generate a CID for itself (as U-type messages require a CID) or b) generate a bot (with a CID) that sends the URES. If no message is receivedwithin two minutes (120 seconds), it means the client has probably failed to properly set up the network for active mode. Ten search results should be sent, to make sure that at least one arrived.

Implementations should, upon failure, indicate as such to the user and fall back to an alternative method of connectivity.

Note that the client should not perform this in all hubs it is connected to. It is up to the client to decide which hub it should do this verification in. A basic implementation may simply do it in the first hub it is connected to.

Note that this procedure must likely be done each time the client is started as the connectivity may have changed during the client's downtime.

Locked