As long as your CPU temp isn't insanely high (especially for long periods of time) you'll be fine, obviously other than the noise lmao. There are some free applications that allow you to see your temp.
Also sounds like a power setting bc you mentioned it only doing it while idling.
Check your bios settings / motherboard settings (on startup) and see if there are any fan settings.
If you have a 3rd party cooler like a watercooled system (mine is from corsair) they have their own applications that control these settings.