Negotiate SEDL (segmented downloading)

Ideas for ADC may be presented here for others to review and point out flaws or further improve the idea.
Forum rules
If you have an account on the wiki, remember to update the ADC Proposals page for new ideas.

http://dcbase.org/wiki/ADC_Proposals_list
Locked
FlipFlop™
Junior Member
Posts: 23
Joined: 17 Apr 2009, 08:29

Negotiate SEDL (segmented downloading)

Post by FlipFlop™ » 06 Sep 2010, 09:20

In private RAR-hubs segmented downloading (SEDL) isn't the most ideal/efficient way of sharing. There's been some discussion about wether it is or it isn't, but even if it isn't, I think a hubowner should have control over in what way he wants to run his/her hub.

The segmented downloading is in no way announced or negotiated with the hub or clients, so hubs that want to enforce normal downloading (and normal slot-occupation) are only left with the option to ban clients that support segmented downloading.

I want to propose the negotiation of SEDL in SUP for client-hub connections.

In favorite hub properties of a hub, an option could be available to disable segmented downloading, it's also possible to disable it according to the hub's ISUP (which wouldn't need a GUI change):

Hub that allows SEDL (no action required) :
Old client: HSUP ... > ISUP ...
New client: HSUP ... ADSEDL > ISUP ... ADSEDL

Hub that doesn't allow SEDL:
Old client: HSUP ... > Kick user: Update your client, it might allow segmented downloading
New client: HSUP ... ADSEDL > ISUP ... RMSEDL > Client will disable SEDL for this hub

Not sure if ADC protocol requires the RMSEDL reply in ISUP, just leaving it out of the reply could be enough.

FlipFlop™
Junior Member
Posts: 23
Joined: 17 Apr 2009, 08:29

Re: Negotiate SEDL (segmented downloading)

Post by FlipFlop™ » 06 Sep 2010, 09:55

Small modification, an 'old' hubsoft might not reply with ADSEDL although it is allowed to be used. That's just because it was never negotiated.

There's only one 'workaround' for that availabe:
Hubsofts supporting SEDL-feature but who need it disabled need to reply with RMSEDL to prevent it's usage.
So clients should only disable segmented downloading in a hub that sends RMSEDL in ISUP.

Big Muscle
Junior Member
Posts: 39
Joined: 01 Jul 2008, 19:27

Re: Negotiate SEDL (segmented downloading)

Post by Big Muscle » 06 Sep 2010, 10:15

FlipFlop™ wrote:In favorite hub properties of a hub, an option could be available to disable segmented downloading, it's also possible to disable it according to the hub's ISUP
I think this will be very complicated to disable segmented downloading per hub, because it could it in all hubs.

FlipFlop™
Junior Member
Posts: 23
Joined: 17 Apr 2009, 08:29

Re: Negotiate SEDL (segmented downloading)

Post by FlipFlop™ » 06 Sep 2010, 13:30

It's enough if a client can announce wether segmented downloading is globally enabled or disabled in the client. It doesn't have to change a client's behavior, just show what feature it uses.

A client with segmented downloading disabled can send: HSUP ... RMSEDL
That way a private hub that would otherwise ban the client for it's segmented downloading capability, can now see that feature has been disabled.

Pretorian
Site Admin
Posts: 214
Joined: 21 Jul 2009, 10:21

Re: Negotiate SEDL (segmented downloading)

Post by Pretorian » 06 Sep 2010, 17:39

FlipFlop™ wrote:A client with segmented downloading disabled can send: HSUP ... RMSEDL
That only makes sense if you have done ADSEDL first.

There are three things that can happen;
HSUP ADBASE ADSEDL
ISUP ADBASE RMSEDL
<client automagically disables segmented downloading etc>
or
HSUP ADBASE ADSEDL
ISTA 148 Segmented_downloading_is_not_allowed_in_this_hub_(48_is_Invalid_feature) FCSEDL
HSUP RMSEDL
ISUP ...
or
HSUP ADBASE ADSEDL
ISTA 248 Segmented_downloading_is_not_allowed_in_this_hub_(48_is_Invalid_feature) FCSEDL
<user is notified about the feature and disables it>
<user (re)connects>
HSUP ADBASE
ISUP ...
The first case require the client to handle "the hub tells me to remove things". The second case require the the client and hub to handle a "recoverable" state. The third case require the client to simply display a message to the user that they need to disable segmented downloading.

I.e., the third case is the simplest to do, and require only clients to implement sending SEDL.

Quicksilver
Member
Posts: 56
Joined: 17 Aug 2009, 21:32

Re: Negotiate SEDL (segmented downloading)

Post by Quicksilver » 06 Sep 2010, 18:10

I still see the major problem in segmented downloading in the DC++ implementation ...

There is no need to disable this / make it negotiable.

I would rather propose something different:

1. Make a larger default segment size... more like start with 16 MiB and possibly reduce the size for slow uploaders -> automatically no segmentation in rar hubs...

2. Try to cache FileChannels (or what ever the handle to the os ressource is called in your language) , what really costs harddisc is closing and opening channels for reading and writing... as this migth trigger the OS to write small fragments on the disc instead of larger several MIB sized portions.

---

We really shouldn't let something like this in the hand of hub operators/owner. The majority there does just not have the knowledge and should not need to have that knowledge btw.
Better make default settings that please everyone!
Fix the implementation of DC++! Be more reluctant to use larger segments / don't make the implementation turn the os caching against itself!

I really don't see why protocol should be introduced to fix implementation!

cologic
Junior Member
Posts: 41
Joined: 21 Jul 2009, 19:34

Re: Negotiate SEDL (segmented downloading)

Post by cologic » 06 Dec 2010, 13:47

Agree with QuickSilver, strongly, on finding this distasteful. His alternative suggestions are reasonable too, but I'm less attached to them.

Big Muscle
Junior Member
Posts: 39
Joined: 01 Jul 2008, 19:27

Re: Negotiate SEDL (segmented downloading)

Post by Big Muscle » 06 Dec 2010, 14:13

Quicksilver's suggestions are good, but I see another problem in it. When you start with big chunk and the first source will be very slow then this chunk can be blocked for long time altough it could be be split to smaller parts and downloaded from more users.

At the beginning, I would suggest to store last download speed per user (and not per connection as it is now) and use it later to determine chunk size. We could also use US from INF but this value can be arbitrary and doesn't have to reflect real speed.

But I don't see a good choice to let hub admins decide whether user can or can't use segmented downloading, because a lot of them will ban it just because they have minimum knowledge about it. Just similar reason as pays for DHT - admins/OPs ban it just because they know nothing about it and have some false reasons for it.

Locked