Browsed by
Category: Basics

Uptime 3: Climate Change

Uptime 3: Climate Change

Data centers with thousands of computers in concentrated amounts of floor space do have to expend enormous amounts of energy keeping things cool.  Your home data center can almost entirely ignore this issue, except where your computers have to be enclosed.

Server Closet.  Or at least, a server IN A closet.

At some point, you will want some of your servers out of sight.  Any machine that provides some service via the network without being needed in front of you is a server.  Home aesthetics will at some point demand that the thing get out of sight.

Your computer’s case has one or more fans that circulate air through it for cooling.  The fan draws in room air, heats it some with heat generated by the components operating inside, and then ejects it back into the room.  A typical room is large enough to absorb this without moving the needle much on the overall room temperature, so the process can continue more or less indefinitely.

The problem you encounter when putting a computer into a closet is, soon after the door is closed the computer is drawing in and heating already rather-hot air, and the temperature in the closet starts rising.  Much over 95F/35C, and you’re going to start having components on your system board begin to behave erratically or fail.

So don’t let things in there get too hot.  Check if it’s heating up steadily in there, and open the door a bit if you have to.  If you can, add a vent at the bottom of the door, and an exhaust fan or two at the top.  If you get a couple of 180mm fans that are designed to be installed is computer cases, you can probably work out how to power them outside a case, and you will find that they are really, really quiet.

Note: however you route your network cables in and out of the closet, be sure the door is not pinching them every time it opens or closes.  Eventually, a conductor in there will break and you will get to 1) do a really “fun” troubleshooting session, then 2) shop for a new network cable. 

Another thing you will want to avoid during the heating season is letting the air get too dry.  If that happens, you will have a tendency to build up static electric charge on yourself as you move around.  You can potentially zap your computers when you touch them, damaging random expensive things inside them. 

If you can add humidity to your environment, do so.  Get the relative humidity to about 50%, give or take 10%.  But (and this is important!) do NOT use a misting humidifier, one that sprays droplets into the air to evaporate there.  Be sure to use a humidifier that evaporates the water inside it, so the vapor that comes out is pure water.  If your humidifier sends droplets of tap water into the air, when the water evaporates, it will let the salts and minerals dissolved in it float down to the surfaces in the room, forming a fine white dust that you will see everywhere.  This dust has the potential to short out connections on printed circuit boards, causing all kinds of very expensive havoc.

Also don’t let the wiring in your server closet get away from you.  Like this guy did.

There are worse than this.  See /r/cablegore

Uptime 2: The Power

Uptime 2: The Power

Your home is your data center.

Maybe this sounds like a stretch but, unless you live a very low-tech existence (like this guy, perhaps?), this is how we all live now in the 21st century.  Oh sure, you are not going to have to have raised floor to accommodate miles of wiring, or forty tons of lead-acid batteries for power leveling, or gigantic Liebert chillers for cooling down hundreds of servers.  Still, it would be a good idea to give some thought to how your environment can be more comfortable for the dozens of computing devices that make modern life tick.  We don’t necessarily have to keep our homes to the strict environmental standards of large data centers.  Still, it pays not to subject our computing devices to too much environmental stress. 

Consider power. If a device works from a battery, which you recharge when you can, then it will be less sensitive to fluctuations in the power that comes out of your wall sockets. But devices that work straight off your line power can be quite sensitive to spikes or sags. Even if they take the power through a transformer (“wall wart”) it probably offers little or no protection from spikes that can damage the equipment.

Batteries directly power the large data centers, while being continuously recharged from line power or generator 

You won’t have a large roomful of batteries through which to pass all your electricity, providing an absolute filter against voltage sags and spikes. But for any of your digital devices that run on AC power out of a wall plug, you need to consider how to condition the power they get.  Though there are many options, the ones I want you to consider are a good surge suppressor and a UPS. 

Surge suppressors are best for:

  • Devices that have some internal battery capacity, e.g. laptops
  • Devices that will not lose data if the power drops — at least, no data that you care about

Not all surge suppressors do much in the way of suppressing potentially damaging surges.  Some are no more than power strips with a marketing makeover.  I use sites like The Wirecutter to figure out which ones are worth my attention.

For devices that have much more severe consequences when the power drops, you should be looking at a UPS.  A UPS is a teeny-tiny version of that roomful of batteries you see above: the line power keeps a battery inside the UPS charged, and that battery is what actually sends power to your equipment.  Consider a UPS for:

  • Desktop computers
  • Servers
  • DVRs
  • Networking equipment – cable or DSL modems, firewalls, switches, WiFi access.

UPS’s are sized in “VA” which means volt-amps.  Think of a VA as a unit of current to be supplied.  The more VA you have, the longer power will last after a utility failure.  But the larger the device(s) being powered, the faster it draws down VA from the UPS, so the less time you get. You can use a larger UPS to get more time or to power more devices.  Remember, for a desktop computer, you’re going to want to power the display, and any attached external hard drives as well.

I typically use a UPS between 750-1000 VA for a desktop computer.  This gives me enough time to finish up what I am doing, or at least get to a decent stopping point before I run out of juice.  If I can shut down my computer on my own terms during a power outage, I count that a win.  But in case you are not home, be sure every desktop and server is using the critical feature of most UPS’s: to connect a data cable and run a small background app that gracefully shuts down the system when the UPS informs it that the batteries are almost drained.  Otherwise, all you will have done by hooking up the UPS is delayed the sudden power failure by a couple of hours.

Another trick I have enjoyed during a few thunder-stormy evenings is using a smaller UPS (maybe around 500-600 VA) to power all my network gear.  The network stuff is less demanding and so lasts longer.  The result is, after two hours with no power from the utility, my server and desktop are dark.  But my iPad and my phone are happily using the WiFi to fetch email, check social media and even watch a little Netflix if I want.  I can even use that UPS to recharge my mobile devices as needed.

Uptime

Uptime

Every one of us has a data center to care for.  Not everyone takes it as seriously as some do.

The mouseover text for this one reads:

The weird sense of duty really good sysadmins have can border on the sociopathic, but it’s nice to know that it stands between the forces of darkness and your cat blog’s servers.

Point being, what’s trivial to you or me is not so trivial to someone.  And if that someone is a member of your household then you need to take it seriously, if for no other reason than shalom bayit

Think about the things a data center does to create a fundamentally good environment for the computers it houses: climate control, power protection, redundancy, fire protection, physical security.  

But Kahomono, I hear you saying, my house is not a data center!  Oh no?  Let’s talk about a job I had a few years ago.  OK, quite a few years.  But still: we were opening a new data center for a major NYC bank.  We had three computer rooms: the Mainframe room had 8 IBM 390s.  The Time-Sharing room had 4 Honeywell DPS-8s.  And the Mini room had about a dozen computers of various makes: Data General, Pr1me, Tandem, Digital.  There were also a handful of IBM PCs floating around, with which nobody was very impressed.  So let’s round up and say that this “Data Center” — and it was surely that — had about 30 computers housed in it.

How many computers in your home now?  Do you even know?  I can say that in a typical home housing a family of four, you probably have… more than in my 1980’s era data center.  40?  Maybe close to 50?  Consider that your phones and tablets, your set-top boxes, DVRs, gaming consoles, “smart home” controllers and endpoints, not to mention every “smart” appliance you connected to your poor overtaxed WiFi, are all computers at least as powerful and capable as that VAX in our Mini room back in the day.  So if you only counted your desktops and laptop computers, you missed the mark by around 90%, is my guess.

And every one of those computers is capable of violating at least one tenet of information security.  (Remember CIA?) 

  • Confidentiality: it could leak information about you and your activities that you would rather it didn’t.  
  • Integrity: It could damage or alter information it holds, making it less useful or even harmful to you
  • Availability: you could lose information you don’t want to lose.  Think emails, tax returns, photos, music collections, movies, saved game progress.

So what do you do about it that doesn’t turn you into that guy in the cartoon above?  More on that to come.

this post originally appeared on Kahomono – It Means Lucky. 

The Wirecutter on 3-2-1 Backups

The Wirecutter on 3-2-1 Backups

3-2-1 is the watchword for how to do backups.  3 copies, on at least 2 different media, and 1 offsite.  I have written about this a lot, as I consider it the most basic of security basics.

If your data is backed up offsite, ransomware can’t get to it, fire and flood can’t get to it.

Now The Wirecutter has thrown its backup hat into the ring.  They might have a few (million) more readers than I do, so I will go ahead and link to them.

I am not a huge fan of their cloud pick, Backblaze.  I have tried it and found it to be unacceptably slow.  But it’s probably the easiest to use for the non-technical user, so my disagreement is little more than a quibble.

I am currently backing up with Duplicati and then syncing my backups to pCloud.  Duplicati is awesome but I can tell you: when it comes to ease of use, it’s no Backblaze!  If you just read that and felt like you were going to enjoy that challenge, I say, go for it.

pCloud is just as easy to use as Backblaze, but it does not offer anything like as much functionality as Backblaze.  But it’s comparable in price, and if you can handle Duplicati, pCloud won’t even make you break a sweat.

Anyway, here’s the TL;DR:  Make. Your. Damn. Backups!

Nukes Inbound to Hawaii! NOT!

Nukes Inbound to Hawaii! NOT!

The word on why we got treated to a false alarm about missiles heading for Hawaii is this:
(over-simplification alert!)

  1. What was supposed to be an internal-only test message got misdirected to the live alert system
  2. When presented with the much-maligned, “Are you sure?” prompt, the operator did what we all do reflexively.

They clicked Yes.

There’s a security lesson here.  Stop and take a breath and read all these prompts.  Clicking OK automatically is the road to ruin.  So many security-sensitive things are prompted like this.  You get this one chance to stay safe.  Take it.

Have a Random New Year

Have a Random New Year

Randomness is important.  You use it in the physical world when you shuffle a deck for a game of cards or roll a D12 for a result in Dungeons & Dragons.  But you need it even more in the digital world, and it’s more difficult to come by.  You need randomness to select one-time-use keys that you share for symmetrical encryption, to select strong passwords or passphrases, to run fair games at things like online poker and casino games.

The problem is, that for all the miraculous things it can do with random input, software is very bad at generating it.  Algorithms are deterministic, even if they are designed to be difficult to predict. When you use a function like RAND() in Excel, or get randomized challenges in low-stakes gaming, you’re usually getting the output of what’s called a pseudo-random number generator (PRNG).  The PRNG takes a numerical value, called a seed, and generates a series of new values from it.  If the seed is known, then the new values are easy to predict.  If the seed is not known, it’s a lot more difficult — but not impossible.  If you reuse the same seed you get the same sequence.  This property can be useful sometimes, for example, if you want to be able to reproduce a series of plays in a game.  But mostly, it’s a very bad flaw in any process that needs randomness.

PRNGs are fine when it doesn’t matter.  But when it matters you need to harness the unpredictability of the physical world.  One great Internet resource, random.org, uses atmospheric noise to generate its random numbers.  At that site, random bits are available anytime you want, in many forms.  Some are free and some are available to paid members.  It’s an important function for the safety of the Internet as a whole, and it’s worth supporting.

Another use of physical randomness is in EFF’s Dice passphrase scheme.  If you read the instructions, you’ll see that they really don’t want you using a computer — which might be compromised — in any step of the selection of a password/passphrase that matters.

Internet companies have to generate thousands of strong keys per second for encrypted sessions.  Cloudflare, for example, found a very groovy way to solve this problem:

[Photo: Dani Grant]

So my New Year’s wish to you: keep it random!

 

Safer Email

Safer Email

Today let’s think about how to be safer using the oldest internet application still in common use: email. Email predates the Web by about twenty years. So when young people accuse it of being “for old folks” (meaning, people like me) I have to admit they may have a point. But email is still far and away the best mode of communication for business correspondence, and for the exchange of personal messages longer than 160 characters.

And long before the web, but shortly after the creation of email itself, spam was born. In addition to being annoying, spam can create some information safety issues. So there are two main things I want you to remember when seeing spam in your inbox: use the spam you get to better train your filter, and never click on any links nor open any file attachments.

All modern webmail clients have built-in spam filtering. Personally, I use Gmail to read my mail, even mail from other domains (such as safer-computing.com). The benefit of using an established webmail system as your mail reader is that the provider’s spam filters have been exposed to billions and billions of emails, and so they are very well-tuned for a low rate of both false positives (when the filter puts a valid email in the spam folder) and false negatives (when it delivers actual spam to your inbox). The less of either, the happier you are with the result.

You train spam filters by identifying both false positives and false negatives for it. For example, in Gmail, there is a “Report Spam” menu option or button in every non-spam folder and a “Not Spam” button in the spam folder. You should make use of these whenever possible. That means occasionally visiting the spam folder to look for those false positives. The more you do this, the less it will be necessary – because the filters adjust their criteria better to the kind of email you get and even to your subjective tastes about what is and is not spam.

One notable subset of spam you always want to be excluded from are the scams. Disney vacations, prizes in lotteries (that you don’t remember entering), gift cards and many more unbelievable windfalls show up in your mailbox by the hundreds each month. As you no doubt know, these are nothing but scams to get your personal information or attempt to extract redemption fees to claim these imaginary prizes. Mark them all as spam.

And of course, there really is no dead Nigerian prince whose family lawyer wants to pay you 20% of $1.6 billion to help them expatriate the money. The only thing that you will get for responding to these is an escalating series of demands for fees to cover the assorted (made-up) mechanics of moving the (imaginary) money and finally (never) paying you. Sending these emails is a crime, and you can report it to the FBI at https://www.ic3.gov/complaint/

Phinally, phishing. Phishing is the sending of emails carefully crafted to look like they come from a legitimate organization, such as a bank, a government agency like Social Security or the IRS, or an employer. The typical phishing email will have a message designed to create some sense of urgency, and links crafted to resemble the links to the legitimate website being spoofed. For example, the email may alert you to a credit card fraud attempt, and the links embedded go to chasebank.com (for example). The problem here is, Chase Bank’s website is really at chase.com. When you go to chasebank.com, which was created by the scammers, you will indeed find the familiar login screen and so on. When you log in through this screen, you will land on the familiar opening screen of chase.com. However, because you logged in through the scammers’ fake page, they’ve snagged a copy of your ID and password in the process. It is easy to do that and then pass your valid credentials along to the real site, so your experience is the same as usual. The fake login page looks very real because the scammers can easily go to the public pages of the real chase.com and grab copies of all the graphics, fonts, content, style sheets and even a fair amount of the programming code needed to make certain pages look and work the way the real ones do. The result is a presentation that even professionals will have a hard time distinguishing from the real thing. It sounds like a lot of work but it pays very well. One single phishing attack in April netted $495K from a Michigan investment firm. And any given phishing email can go to millions of users at a time.

The lesson here is, never click on links in emails, unless the senders are personally known to you, or for things like password resets that you know you initiated within the past few minutes. Certainly, for financial and government services, you should navigate to their websites by way of known links you have previously saved as bookmarks or stored in secure password-manager records. If you use a search engine to make initial contact with an agency or company, make sure that you skip past the sponsored links and click only on the most relevant non-sponsored one. Phishing emails, like all scams, should be reported to the FBI at https://www.ic3.gov/complaint/.

Whether it’s spam or phishing when an email arrives that “wants” you to click on its links, leave it wanting. Especially, never click on “unsubscribe” links in spam email. Doing that simply confirms for the spammers not only is your email address valid, but you actually read their email. They will reward this by showering you with much love. And spam. Well, mostly spam.

 

A Modeling Job for You

A Modeling Job for You

.

Motherboard, a part of Vice magazine, has published a very good Guide to Not Getting Hacked.  It’s also available as a PDF.

One of my favorite sections draws from the EFF Threat Modeling page.  “Threat modeling” may sound like something a management consultant would explain to you with 19 PowerPoint slides for only $45,000.  But it really just consists of considering these five questions:

  1. What do I want to protect?
  2. Who do I want to protect it from?
  3. How bad are the consequences if I fail?
  4. How likely is it that I will need to protect it?
  5. How much trouble am I willing to go through to try to prevent potential consequences?

Ultimately the goal of information security is not to protect the information assets absolutely.  Protecting anything absolutely is not even theoretically possible.  What we’re trying to do here is, make the information assets more trouble to attack successfully than they’re worth.  If stealing a new sprocket design from the engineers at Spacely Sprockets is worth $4 million, then we have to make it cost an expected $4.5 million or more to get.  That way, even success is failure for the attacker.

But if preserving that design is worth $4 million to us, we’d be idiots to spend $4.5 million defending it.  We could post it on Facebook and save ourselves $500,000.

Threat modeling is really just taking a breath, refusing to panic, and applying all-too-UNcommon sense.

Internet of Crap

Internet of Crap

Welcome to the wonderful world of the Internet of Things. You’ve probably seen this term in the news a bit lately. Perhaps you read about it in connection with a massive botnet called Mirai, or it’s even more potent descendant, IoT_reaper.

The term Internet of Things (IoT), refers to items – other than computers, tablets or mobile phones – that are connected to the Internet and communicate back to their manufacturers or distributors. A prime example of this is, printers and copiers that provide supplies consumption and problem diagnostic data back to the manufacturer. This allows service calls and supply replenishment to arrive with minimal delays in production. A great benefit, to be sure.

The problem arises when large numbers of consumer devices start using this same capability, but without much in the way of careful design or attention to the possible security compromises. A buyer of a $1,500,000 production printer may safely assume that some attention has been given to this issue by the manufacturer. They also know that $1.5M worth of business gives them quite a bit of leverage to press the manufacturer to fix it if something is wrong. But a buyer of a $20 “smart” light bulb has neither of these safety factors. For $20, you get what you get.

As more low-cost consumer devices all start turning up with internet capability, we start to see some very odd ideas expressed in this technology. Late in 2015, we learned about a vulnerability in Samsung refrigerators that exposed customers’ GMail logins (including passwords) to cyber-criminals. Many people had questions about this. “How could this happen?” “Have they fixed the problem?” My question was, “WTF were REFRIGERATORS doing with GMail logins?”  This illustrates the first principle of IoT security

  • 1st Principle of IoT security: Don’t give your devices information they don’t need. Think about what could be the impact, when information you give to something like a refrigerator is leaked to cyber-criminals. If a device works and does what you want despite the fact it’s still asking for some information, drop the matter. Its feelings won’t be hurt; it has no feelings.

As I have said a number of times in this space, the essence of security is not absolute, but relative safety. Make trade-offs intelligently between risks and benefits.

When I get a new device, one of first things I do is assess what I will gain by connecting it to my network and to the internet, vs. what might be at risk if the device’s security is not up to snuff. Most of the time, my conclusion is, “don’t connect it at all” or “connect it to the home network but keep it off the internet.” If your router has a parental controls feature, where you can restrict your kid from getting online, you can also use that to restrict your fridge from getting online. Most devices’ main reason for being connected to the Internet is to feed data back to its manufacturer that can — at the most benign end of the spectrum — be used for marketing purposes.  Consider that when assessing the risk side of this question.

  • 2nd Principle of IoT security: Don’t allow devices to connect directly to the Internet or the rest of your home network unless necessary.  Figure out what you’re really giving up if you don’t connect the device. And if the answer is, “not much”? Don’t plug in the wired connection, don’t give it the WiFi password, just say no.

Brian Krebs is an information security researcher (hacker!), with a blog that is very popular in our field. He does a lot of independent investigation of cyber-criminals, and as a result he often draws their ire. He has had heroin shipped to his door, and they have spoofed phone calls to police that result in the SWAT team being dispatched for the non-existent “hostage situation.”

Last fall, Krebs’ blog website was attacked by the largest denial-of-service that had ever been seen to that point: a botnet directed over 660 gigabits/second of bogus traffic at his server. For comparison, the fastest connection available from Time-Warner in Rochester is 50 megabits/second, so this was larger by a factor of 13,200. All of that focused on a single web site will disable the servers just because of the volume.

Upon investigation, the source traffic was found to have been infuriatingly simple. The attackers had just scoured the internet for connected IoT devices and checked them to see if they still used the manufacturer’s default username and password to allow remote access. They were able to find millions that did, mostly CCTV cameras and cheap routers. Those were harnessed by the criminals to start sending Krebs a synchronized tidal wave of garbage network traffic. It’s tempting to say they were “hacked” but they weren’t, really. Their owners had offered them to the public with the documented default logins, effectively free to use for all comers.

  • 3rd Principle of IoT security: Change the default username and password. If the install process forced users of all new devices to choose any non-default username and password, that alone might have been sufficient to stop the attack on Krebs.

So to recap: our three principle of IoT Security are:

  • Don’t give your devices information they don’t need.
  • Don’t allow devices to connect directly to the Internet or the rest of your home network unless necessary.
  • Change the default username and password.

Yes, there are problems in IoT security, and we’re going to need the manufacturers to address poor designs and worse implementations. But by applying these three principles, we can reduce the impact on our own lives, so that we still get some benefit from these modern things.

 

Time to Go!

Time to Go!

Where?  To the Rochester Security Summit of course! It kicks off tomorrow for two days of security geeking-out.  I am looking forward to it plenty.  My talk is on Friday at 2PM about full and responsible disclosure of bugs, bug bounties and so on.

This weekend I will make a post here, covering that topic.