You've probably tried to install a 800K floppy drive instead of the 400K drive it came with.
The Macintosh contained a 400 KB, single-sided 3.5-inch floppy disk drive and dedicated no space to other internal mechanical storage. The Mac OS was disk-based from the beginning, as RAM had to be conserved, but this "Startup Disk" could still be temporarily ejected. (Ejecting the root filesystem remained an unusual feature of the Classic Mac OS until System 7.) One floppy disk was sufficient to store the System Software, an application, and the data files created with the application. Indeed, the 400 KB drive capacity was larger than the PC XT's 360 KB 5.25-inch drive. However, more sophisticated work environments of the time required separate disks for documents and the system installation. Due to the memory constraints (128 KB) of the original Macintosh, and the fact that the floppies could hold 400 KB, users frequently had to swap disks in and out of the floppy drive. For that reason, external floppy drives were frequently used. The Macintosh External Disk Drive (mechanically identical to the internal one, piggybacking on the same controller) was a popular add-on at US$ 495. Third-party hard drives were considerably more expensive and connected to the slower serial port (as specified by Apple), though a few manufacturers chose to use the faster non-standard floppy port. The 128K can only use the original Macintosh File System for storage.
A stock Mac 128K with the original 64K ROM cannot use Apple's external 800 KB drive with HFS, nor Apple's HD20 hard drive. A Mac 128K that has been upgraded with the newer 128K ROM (A Macintosh 128Ke) can use internal and external 800 KB drives with HFS, as well as the HD20.