It is a corruption of "bite" and comes from it being the largest "bite" of data a computer could take at a time.

1956, Amer.Eng.; see bit (2). Reputedly coined by Dr. Werner Buchholz at IBM. http://www.etymonline.com/index.php?term=byte Expansion of bit, coined by Dr. Werner Buchholz in July 1956, during the early design phase for the IBM Stretch computer. http://en.wiktionary.org/wiki/Byte 1. unit of computer information: a group of eight bits of computer information, representing a unit of data such as a number or letter 2. computer storage unit: a unit of computer memory equal to that needed to store a single character [Mid-20th century. Probably alteration of bit after bite "morsel," or acronym <binary digit eight] http://encarta.msn.com/encnet/features/dictionary/DictionaryResults.aspx?refid=1861593835 (BinarY TablE)- BYTE- The common unit of computer storage from desktop computer to mainframe. It is made up of eight binary digits (bits). A ninth bit may be used in the memory circuits as a parity bit for error checking. The term was originally coined to mean the smallest addressable group of bits in a computer, which has not always been eight. A byte holds the equivalent of a single character, such as the letter A, a dollar sign or decimal point. For numbers, a byte can hold a single decimal digit (0 to 9), two numeric digits (packed decimal) or a number from 0 to 255 (binary numbers). http://encyclopedia2.thefreedictionary.com/byte