See more here:

Troubleshoot Activation Problems

Machine Quota exceeded: 

If the number of machines on which you're using Katalon Studio exceeds the number of licenses that you purchased, you have two options:

• Subscribe to more licenses to cover more machines and execution sessions. See Upgrade Subscriptions.
• Remove the machines. See Revoke and transfer a license.

Another workaround: 

The message "Machine Quota Exceeded" occurs when

KSE: The number of machine IDs is higher than the number of licenses

KRE: The number of sessions is higher than the number of licenses.

Solution:
Remove all machine IDs in the Registered Machines block under the Katalon Studio Enterprise or Katalon Runtime Engine tabs from the Admin Page, and try to execute tests or activate Katalon Studio again.