kernel: optimize the necessary drive for loading [1/8]

PD#SWPL-96062

Problem:
remove pinctrl and clk drivers unrelated to the project

Solution:
remove pinctrl and clk drivers unrelated to the project

Verify:
sc2

Change-Id: I99c5718da9dec81f2eb3ffca3ed432201efdedeb
Signed-off-by: Wanwei Jiang <wanwei.jiang@amlogic.com>
6 files changed