Now I have an answer thanks to Mahesh Holenarasipura at SAP.
First set the Roles and Access to give you want you want in the system. For us that means the role access control for the activities is set to unrestricted read and restricted write. That allows our team to see all the activities at the customer location. It will only allow them to change those that are theirs or they are involved in.
These settings do not work for the report. However, by creating the report in the Business Analytics and setting the employee selection to a relative select we can accomplish our reporting needs as well. One report for everyone, and it works perfect.
Thanks Mahesh!