Could well be the condensor in the mag, they usually take a little longer to warm up and fail tho when they start to go. Check it out, if it is an original one replace it. You can wire a new condensor temporarily in position outside the mag, it just needs to be wired into the feed and earthed. See if it makes a difference before pulling the mag and stripping it down. New condensors are cheap, Marks Magneto will sell you a couple for a few bucks each.
Also check your manifold seals. Does the bike run on choke well but not idle down on run ? Spraying a little wd40 or cold start around the join will tell you if it is sealed well, as if it changes the running, you have an issue there.
Let us know what you try, and what you find out.