This feature by design. We have to create contact to store references and make it searchable. You can always delete contacts from the system that has zero tickets associated with them inside Manage >> Contacts. Learn more about managing contacts.