Similar to the architecture of a building, the architecture of a computer refers to the design structure of the computer system and all its details: the system, the circuits, the chips, the busses, the expansion slots, the system firmware, BIOS, etc. The architecture largely determines how fast the computer is and what it can do. It also decides whether one computer is compatible with another. Can the same boards be used? Yes, if the architecture is compatible. Different models will have basically the same uses, but with varying degrees of performance. The architecture is what ensures backward compatibility, which means that your old software can run on a new computer.
If a computer has an open architecture, it means the company that builds the computer has published the specifications of the computer's design. Open architecture lets third-party developers create add-in boards (for customizing your computer, assuming the computer has expansion slots) and other add-on hardware that will work in the machine. So, open architecture means that any other company can create memory boards, video adapters, hard disks, and so on that function properly in the computer. An open architecture even allows other manufacturers to build computers that work identically. The Apple II and the IBM PC have open architectures, which is why so many clones (copies that work the same but usually cost less) of these machines exist.
If a computer has a closed architecture, the specifications for creating it are not public. That makes it difficult, if not impossible, for anyone else besides the machine's original manufacturer to create peripheral hardware to add-on to the machine. It also means no one else can recreate the same sort of computer. The Macintosh has a partially closed architecture-Apple released enough information for other manufacturers to build addin boards and peripherals, but not enough for them to build Mac clones.
The terms "open" and "closed" architecture can also simply refer to whether or not a computer has expansion slots inside of it that allow you to plug in other add-in boards to expand or customize your machine. The original Macintosh had, in this sense, a closed architecture, whereas the later Macs have an open architecture.