keycloak-js How to authenticate AND authorize users - keycloak

Question

I'm having a bit of trouble understanding how to link together authenticating and authorizing users in my frontend using Keycloak and the keycloak-js adapter.
To preface this: I know that the keycloak-js adapter requires the client to have public as the Access Type.
My main question is how can I use Keycloak authentication AND authorization via the same client. If the Access Type MUST be set to public for the keycloak-js adapter to work, then how can I restrict access to my API depending on the users permissions?
Will I need two clients for this? One for authentication and one for authorization? If so, how can I use the access token from the authentication client in the authorization client?
My current setup in Keycloak is as follows:
Realm: admin-service
Client: admin-service-api
Access Type: confidential (because I want to use authorization to restrict api requests according to permission levels)
Root URL: http://localhost:8080/
Valid Redirect URIs: http://localhost:8080/*
Roles:
admin
Users:
test-user
My authorization setup for admin-service-api is as follows:
Resources: Books Resource
Uri (these are my API endpoints):
/v1/books
/v1/books/{id}
Scopes:
books:delete
books:create
books:update
books:read
Policies:
Default Policy
Books Policy:
Realm roles:
admin (the required checkbox is not checked)
Permissions:
Default Permission
Books Resource Permission:
Resources: Books Resource
Apply Policy: Books Policy
So with this setup, I have restricted my API (which is written in GoLang) to only allow requests if the requesting user has the appropriate permissions by making a request to the Keycloak API via https://my.auth.server/auth/realms/{{realm_name}}/protocol/openid-connect/token to retrieve the users access token and a list of the users permissions.
From there I can use this access token to make requests to my API to create/read/update/delete books so long as my test-user has the admin role. If my user does not have the admin role, the user is presented with an unauthorized message (401).