This was originally posted as a Slashdot comment. It discusses the idea of prioritizing traffic in an ISP environment, ideally using markings generated by the customers.
I do network engineering at an ISP. We are small, though I have discussed these things with my peers at larger networks.
Once you scale above a very small network (like your home connection), allowing congestion isn’t really okay in practice, even with QoS. When I say it’s not “okay” here, I’m speaking purely technically.
It might be possible to let networks congest somewhat if you had a large amount of elastic traffic that you could reliably identify. Netflix, for example, could meet these criteria. But that’s not okay politically; that’s an example of why net neutrality is good!
QoS in carrier networks is only useful for priority (de-)queuing of traffic to reduce latency and jitter. For example, real-time voice or video traffic could benefit. This is where it’d be nice to actually be able to honor user traffic markings.
It’s not (currently at least) practical to make the decisions on a flow-by-flow basis in the core of the network (which is what your proposal would require). This is a hardware scaling issue. To be clear, tracking flows statistically is okay at scale. ISPs do plenty with NetFlow/sFlow. But taking an incoming packet, assigning it to a flow, and marking it appropriately, for every packet, in real time is the scaling challenge.
The following approach would scale perfectly in trusted CPE (ONT/cable modem) or reasonably well in a DSLAM (for DSL). Give each user (for example) two queues. Honor the incoming DSCP markings. Put a small, but reasonable, limit on the size of the priority queue; overflowing traffic gets remarked and placed into the non-priority queue. Then, honor markings through the rest of the network.
There are a few problems with even this approach. First off, there are going to be users who legitimately create more high priority traffic than any limit that’s acceptable across the board. Is it okay to charge them for a higher limit? If not, how do you avoid gaming the system? If yes, won’t that incentivize ISPs to set the limit to zero and charging for all priority? Is that okay? If so, what fraction of people will request and pay for priority in that world? Will that be enough to encourage application developers to mark traffic appropriately? Or does this just degrade into our current zero-priority Internet?
Second, this only gets you one direction (upload). To handle the download direction, you’d need to honor priority bits on your upstream and peering links. But there, you can’t trust the markings (unless it’s a 1:1 peering link and you are guaranteed your peer implements a compatible policy at their incoming edge), at least without policing. Policing the queues there is easy, but gives you terrible results in real life. If the limit is exceeded with traffic that “should not have been” marked priority, it will destroy the prioritization of “legitimate” priority flows by forcing some fraction of their packets into the non-priority queue. If you accept all (or just a high enough fraction of) incoming traffic as priority traffic, then you have destroyed the prioritization yourself. If you try to mark flows per IP/customer, we’re back to that scaling problem.
It might be possible to do something that involves tracking flows at the customer edge and using the incoming markings for the downstream direction. But this is only prioritizing in the last mile. At best, this is a lot of work for very little benefit.
I’m documenting this mainly for myself, but if you’re ending up here based on a Google query, I hope it helps!
I tried to upgrade our Nexenta storage system (currently running 4.0.3FP3). After apt-get downloaded packages, I received this error:
Download complete and in download only mode
Upgrade is in progress. Please DO NOT interrupt...
Creating Upgrade Checkpoint...
Feb 03 19:13:23 EXCEPTION: FormatError: Failed to parse menu.lst: section content not complete
Uncaught exception from user code:
com.nexenta.nmu.FormatError: Failed to parse menu.lst: section content not complete
at /usr/perl5/5.12/lib/NZA/NMUUtil.pm line 731
NZA::NMUUtil::_mark_rootfs('syspool/rootfs-nmu-008', 0, '') called at /usr/perl5/5.12/lib/NZA/NMUUtil.pm line 817
NZA::NMUUtil::clone_rootfs() called at /usr/bin/nmu line 526
Nexenta tech support found that the issue was empty BOOTADM blocks in /syspool/boot/grub/menu.lst:
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
The fix is to remove those and run
bootadm update-archive -v.
If I understood correctly, the cause may have been using
beadm destroy in the shell instead of
setup appliance checkpoint ... in nmc.
This is another Facebook comment being kept here for posterity:
Regarding H.R. 1076…. I just read the text of the law and one source for the NRA’s position on it: http://www.americas1stfreedom.org/articles/2015/11/20/using-the-terrorist-watchlist-against-gun-owners/
The NRA seems to make some points worth considering:
1) Are terrorists (as defined in this law, and known to be such at the time of the purchase) actually buying guns from legal sellers?
I share their skepticism. If this isn’t a realistic problem, then there’s no point for the law and everything else is moot. So supporters need to prove that point first.
2) The NRA is saying this is based on the terrorist watch list, which is a mess: “Consider, for example, that even three federal legislators, including Sen. Ted Kennedy, found themselves on the list. As Charles C.W. Cooke pointed out on nationalreview.com, some 280,000 people on the list have ‘no affiliation with known terrorist groups’ but simply fall under ‘reasonable suspicion.'”
I didn’t parse the text of the law enough to know if using the terrorist watch list is what would actually be happening here. The text talks a lot about the attorney general denying a transfer, but would that be implemented in practice by blanket denying based on the watch list? It certainly could be. And it sure seems like that’d be a lot easier than trying to create a separate subset list of “terrorists to not allow to buy guns”. Plus, if they did create a separate list, there’s a potential for backlash if they miss someone who is on the bigger list, so that factor will encourage the use of just one list.
I think it’s widely agreed that the watch list is problematic in many ways. That’s why certain people have to deal with TSA redress numbers, etc.
3) There are essentially no consequences for listing someone.
As far as I know, this is generally the case with a lot of laws, so I’m not sure whether that’s creating a new or bigger problem here specifically.
Overall, I don’t see a lot of point for the bill. If this actually is a problem, the bill doesn’t seem too terrible to me. Ideally, I’d like to see the government be required to pay your court and attorney costs if you prevail on a challenge to your being listed.
I wrote the following comment in response to a Facebook post about religious freedom laws, which have been in the news lately.
I appreciate the argument that people should not be forced into conducting business with others, but I’m not sure if any bright line rule works well here. Some things to contemplate:
Is it the right balance to allow a gas station owner to refuse to sell gas to gay people, black people, members of another religion, etc.? Does your answer change if all the gas station owners in the same town feel the same way? (For example, if the result is Muslims can’t buy gas in a town.) What if we scale up to the vast majority of the gas station owners in a county, half a state, or more?
Should the threshold should be different for “essentials” (food, fuel, housing, etc.) than for optional things (wedding cakes)? If so, what’s essential? Is Internet access essential? What about cable TV? Maybe they’re separately special because they tend to be natural monopolies? What about the one formal wear business for a hundred miles?
In regards to the cake examples… One possible answer is that speech is different than products. For example, we might say: yes, we should prohibit the black baker from refusing to sell a cake solely because the buyer is a KKK member, but we will not compel him to write a message on it with which he disagrees.
If so, is the baker’s free speech right absolute? Or if he is willing to write “Congratulations!” on a cake for some customers, can he be compelled to use the same text for anyone (including for gay weddings, graduating from seminary of another religion, etc.)?
What about pharmacists and various types of birth control? Or doctors and abortions? Or doctors refusing patients on (non-religious, public health grounds) who are anti-vaccine for non-medical reasons?
To come full circle, can the gas station owner refuse to sell to people with tattoos for non-religious reasons (because he associates tattoos with gangs)? What if he just thinks tattoos are stupid? If religious reasons are special, who decides what is a legitimate religious interest and what is a legitimate religion?
I get this question a fair amount. Here is the latest version from someone on Facebook: “To all the computer wizards out there: Which antivirus do you suggest putting on a computer?” My response…
Short version: Spend your money on backups instead.
I think I am in the minority in the industry on this, but I tend to recommend “none”. Antivirus software is a bit like insurance. You are going to pay something (dollars, at least some slowdown, and potential problems) all the time to potentially avoid paying a big something later.
No antivirus software can completely protect you from reckless behavior, and if you’re responsible, the risk is probably acceptably low. By “responsible”, I mean things like: don’t open attachments or click on links you were not expecting, regularly apply updates to your OS and browser, and only install software from trustworthy sources.
For the typical home user, the cost of recovering from a virus is pretty minimal, if you have backups. That is where you should focus your energy and/or money. And always test your backup method to make sure you can actually get your files off of it!
Update: Running Microsoft Security Essentials is probably reasonable. It’s free; you get it as part of Windows Updates.