Yes there seems to be so many ways in which the user.lib file can cause problems if not kept clean, so thats why I always start fresh with each new course and occasionally reset it at other times too. Lez brings up some ways in which the user.lib can cause problems in his APCD video series, and how to keep the library clean, which is what made me do it in the first place. The only benefit to not resetting it would be to access a lot of textures you've used in the past, but I keep the textures I've used as TGA files on my computer rather than within APCD, and import them individually when I wish to, which keeps things clean. I have never had a large user.lib file, but based on what Ive seen of other designers sometimes having issues with their textures, and in general having some experiences with other shortcomings of APCD, I strongly suspect that having a lot of textures in your library file is a bad thing.
Unless you enjoy running into problems...
Sometimes you can solve certain problems that have already occurred by deleting the user.lib too, like if a texture on my course has gotten corrupted and displays oddly both in APCD and in Links (which has happened to me 3 or 4 times), or if my seam blend properties get mixed up (which has probably happened to me 5-10 times despite not displaying panorama within APCD), i can solve them quickly by deleting my user.lib, then opening an earlier version of my course before the problem happened, then opening the newest version of the course again. Of course you have to be mindful about any changes to specific textures you made on purpose in between though.