Let $I=\int x\cos^{-1}x\,dx$ $=\int\cos^{-1}x\cdot x\,dx$ $=\cos^{-1}x\int x\,dx-\int\left(\frac{d}{dx}\cos^{-1}x \int x\,dx\right)dx$ $=\cos^{-1}x\frac{x^2}{2}-\int\frac{-1}{\sqrt{1-x^2}}\cdot\frac{x^2}{2}dx$ $=\frac{x^2}{2}\cos^{-1}x+\frac{1}{2}\int\frac{x^2}{\sqrt{1-x^2}}dx$ $\therefore I=\frac{x^2}{2}\cos^{-1}x+\frac{1}{2}I_1$ ...(i)where $I_1=\int\frac{x^2}{\sqrt{1-x^2}}dx$Put $x = \cos\theta$ $\Rightarrow dx = -\sin\theta\,d\theta$$\therefore I_1=\int\frac{\cos^2\theta(-\sin\theta)}{\sqrt{1-\cos^2\theta}}d\theta$$=-\int\cos^2\theta\,d\theta=-\frac{1}{2}\int(1+\cos2\theta)d\theta$$=-\frac{1}{2}\left(\theta+\frac{\sin2\theta}{2}\right)+C$$=-\frac{1}{2}\left(\theta+\frac{1}{2}\times 2\sin\theta\cos\theta\right)+C$$=-\frac{1}{2}\left(\theta+\cos\theta\sqrt{1-\cos^2\theta}\right)+C$$=-\frac{1}{2}\left(\cos^{-1}x+x\sqrt{1-x^2}+C\right)$... (ii)From (i) and (ii), we get $I=\frac{(2x^2-1)\cos^{-1}x}{4}-\frac{x}{4}\sqrt{1-x^2}+C$