Because you keep running the AC.

Nuclear fission reactors do not have to be “so big.” This is the SNAP-10A nuclear fission reactor, thermoelectric generator, which generated 500 watts to power a series of military missile surveillance radar satellites in the 1960s. The reactor is the little can at the top, the rest is space radiator:

This is the Kilopower KRUSTY reactor that is currently in development as part of a 1,000 watt thermoelectric generator for space applications:

In both these reactors, heat is conducted across a thermopile to produce electricity, just like in an RTG, but the heat comes from a moderated fission reaction, not passive radioactive decay as in earlier RTGs. No shielding is needed until such a reactor is activated, and since it’s meant to be used in space and away from people, none may be needed there either.

This is the CROCUS research reactor in Switzerland, which you can get the scale of from the labels:

It generates no power and 100 watts of heat, and is used to produce neutrons and conduct experiments. It needs shielding to protect people from its radiation, but is simple and small and runs at low temperature with minimal risk to anyone.

Small fission reactors are widespread, but they don’t produce much power. You need to run your AC, so we need something bigger.

This is a modern naval reactor sealed inside the shielded containment vessel that will keep it safely away from people while in service and for long after. It generates 700 megawatts of heat, which will turn steam turbine producting something like 350,000 shaft horsepower and 125 megawatts of electricity.

But you need to run the AC, so we need something bigger:

This is the containment building for a new 1-gigawatt reactor under construction in Georgia, which will produce 3,000 gigawatts of heat to make enough power to keep the AC humming in a quarter of a million homes. To produce that much heat, it needs many tons of nuclear fuel, inside a massive containment and shielding structure, supported by an array of giant pumps, valves, pipes and controls and enormous cooling towers to carry away excess heat…

And of course, gigantic turbines and generators to convert the heat into electricity:

The plant produces 8 times the electricity from 4 times the heat. This is in large part because naval reactors are designed to be small and civilian reactors are designed to be economical.

A 250kW research reactor

Nuclear reactors can be built arbitrarily small, the reactor above would fit into a truck, perhaps even a minivan. Somewhat larger systems power nuclear submarines and let’s say medium sized reactors power several aircraft carriers, there were also nuclear-powered icebreakers built. In 1960s USA built and tested nuclear-powered aircraft and missiles, Russia is now trying to rediscover the concept. You could build a nuclear reactor to power a wristwatch, at least in principle.

It’s just that the economics of a nuclear reactor as a power source for a country dictate a large reactor. You can build small reactors, but for a reactor that’s 10% the power you’ll need considerably more than 10% the money, probably closer to 30% or so (exact numbers depend on many factors, including size). That’s why with nuclear power you need to go big, not because of any technical or technological limitations, but because of economic constraints. That wristwatch better be a Rolex.