I strictly play utilizing time travel, so I'll do what I can to help.
First things first, you really needn't worry so much that your hands shake when adjusting for Daylight Savings Time. Putting the 3DS clock back or forward an hour will have absolutely no ill effects on your Animal Crossing town. If you switched the date by weeks, months, etc., that's when you'd be risking something, but even then you can prevent anything bad from happening.
There are some things I can't say much on. I can't speak to why people insist on using the 3DS clock, because I personally only ever time travel by talking to Isabelle while loading up the game. I also can't speak to the Stalk Market because I've never bothered to try it out because I don't want to potentially waste money if Reese isn't offering much. I also can't really speak to traveling backward in time other than in a narrow, specific manner.
You didn't clarify what your two reasons for wanting to time travel are, so it's difficult to actually go into much detail. That said, there are a few things to say:
1) You possibly already know this, but rather than midnight, every new day in New Leaf technically starts at 6 AM.
2) If you're worried about any of your favorite villagers moving away, your flowers wilting or weed growth, there are countermeasures against these things.
- To prevent any flowers from wilting if you want to go more than a day or so ahead, set the Beautiful Town ordinance as your town ordinance if it isn't already. Note that the changed ordinance doesn't go into effect until the next in-game day at 6 AM, so if you need to make it your ordinance, remember to go to the next day before doing the heavy duty time travel. As for weeds, Beautiful Town should keep them relatively under control. I still get some sometimes, but they're manageable and less than would appear without the ordinance in place.
- To prevent any of your favorite villagers from moving out, be patient and wait until one of your villagers pings and mentions that they're thinking about moving out. Either allow them to leave or tell them to stay depending on how you feel about them. Regardless of your choice, you should now be free to time travel however far ahead you desire without anyone leaving unexpectedly. I can't promise that this is 100% guaranteed to work every time, but I have personally used this method to repair grass erosion in my town by going ahead a month.
3) In case this is relevant, my method of play involves time traveling through multiple in-game days per real day. I do whatever I want in the town for the day like checking the stores, going to the island or whatnot, saving and quitting, talking to Isabelle, and having her switch to the next in-game day. This method is absolutely safe as long as you keep an eye on things. Talk to all of your villagers to make sure there are no rumors about anyone wanting to move out and that no one pings you about moving out themselves. Water any flowers if need be, like if you're growing hybrids or didn't want to use Beautiful Town.
4) Earlier I stated that I'm not very familiar with backward time travel except for one specific type. Here it is: If one of your reasons for wanting to time travel is that you can't play every day or want to safely put the game down for a week, numerous weeks, or maybe months, there is a way to keep your town absolutely pristine. If you keep track of each time and date that you last saved and quit the game, you can always time travel back to that exact date and time and the game will regard it as the same exact day with anything you might have bought at the stores still sold out, flowers still watered, errands you need to run still available. I've taken numerous sabbaticals from the game over the years since its release and tend to utilize this a lot.
5) Should you ever find yourself wanting or needing to replace a villager out of boredom or whatever else and you decide you want to plot reset, and let's say you spend hours plot resetting and the villager won't plot in a good spot? Assuming you played the previous in-game day, you can load up your mayor and time travel back to the previous in-game day at, say 5:55 AM or so, and then save and quit, wait however many minutes until 6 AM rolls back around, and resume plot resetting. I've done this multiple times using the in-game clock while numerous people have said to use the 3DS clock, so honestly I'm fairly sure it's a matter of preference rather than any hard rule.
There's more I can probably say about it but this is what seemed pertinent. Also, I know I kind of covered some stuff that might not have fit what you're looking for since you said it has nothing to do with villagers moving in or out, but aside from flowers wilting there don't seem to be that many issues that would permanently "damage" a town that would arise from time travel if villagers aren't an issue.
I hope this has been at least somewhat helpful. If you need any more clarification about anything I'd be happy to say more. If your interests are more to do with going back in time outside of the narrow purview I laid out, someone else will have to help out with that, and it appears that perhaps someone just did.
Best wishes.
