Most of today's computers are serial. Their memory contains a large array of instructions, and they obey these one at a time. This is true of PC's, Macs, the University's main computers (a Vax and a Dec 5500), etc. One can also build parallel computers, which have more than one processor and so can obey more than one instruction at a time. These are less common, although big parallel computers are widely available as research facilities: Connection Machines, Crays, etc.
The design of parallel computers varies widely between machines, and I won't say more about them. Serial computers are less varied. Here's an account of the main components of a typical serial computer.
Consists of a sequence of words - nothing to do with words in English! Words are just sequences of binary on-off switches, each switch being able to store one binary digit or bit. All words contain the same number of switches (usually 16 or 32). Altogether, the settings of the switches in a word make up a binary numeric value.
Each word in memory has its own address, determined by the electronics. The addresses run sequentially, usually from 0 upwards.