“hardly working” means something like “they just work a little bit”

you’ve probably meant “working hard at” (or similar).

hard -> hardly; meaning becomes different.