In Licensing GSP Deployments by Terminal Command , I covered the officially sanctioned way to license GSP5 for Macs.
I believe that was still working even as of 10.13.5. It certainly was working in 10.12.
Unfortunately, with 10.13.6—unless you had an already activated GSP when you upgraded to 10.13.6 from 10.13.5 or earlier—you might get error messages like
So I opened a ticket with McGraw Hill, and they said:
Lo and behold, that was the the old non-sanctioned, unofficial way I used to license GSP5 on Macs.
The license for all users is stored in the "/Library/Application Support/The Geometer's Sketchpad/" so it must be a permissions issue with that folder, either creating it for a new install, or accessing it when deregistering an old install.
If you copy the "The Geometer's Sketchpad" folder for a non-admin user install (which would be found in "~/Application Support/The Geometer's Sketchpad", to the global /Library/Application Support/ folder, that should also work, provided that the folder permissions allow read/write.
Tested it out on 10.13.6. Works!
So, yeah, use the old way until McGraw Hill figures out what's going on.