brcm63xx: remove gpio base workaround