1)<img src="https://tex.z-dn.net/?f=a%2Bb%3D5%5Csqrt%203%20%5C%2C%20%2C%20ab%3D16" />
elevemos al cuadrado la suma
<img src="https://tex.z-dn.net/?f=%28a%2Bb%29%5E2%3D25%283%29%5C%5C%0Aa%5E2%2B2ab%2Bb%5E2%3D75%5C%5C%0Aa%5E2%20%2Bb%5E2%20%2B%202%2816%29%20%3D%2075%5C%5C%0Aa%5E2%20%2Bb%5E2%20%2B%2032%20%3D%2075%5C%5C%0Aa%5E2%20%2Bb%5E2%20%20%3D%2043%0A" />
2)<img src="https://tex.z-dn.net/?f=%28a%2Bb%29%5E2%3D10%5C%2C%20%2C%20a%5E2%2Bb%5E2%3D6" />
desarrollemos el cuadrado
<img src="https://tex.z-dn.net/?f=a%5E2%2B2ab%2Bb%5E2%3D10%5C%5C%0Aa%5E2%2Bb%5E2%2B2ab%3D10%5C%5C%0A6%2B2ab%3D10%5C%5C%0Aab%3D2" />
3)<img src="https://tex.z-dn.net/?f=a%2Bb%3D%5Csqrt%7B15%7D%5C%2C%20%2C%20ab%3D6%20%5C%2C%20%28a%5E2%2Bb%5E2%29%28a%5E3%2Bb%5E3%29%3D%3F%0A" />
<img src="https://tex.z-dn.net/?f=%28a%2Bb%29%5E2%3D15%5C%5C%0Aa%5E2%2Bb%5E2%2B2%286%29%3D15%5C%5C%0Aa%5E2%2Bb%5E2%3D3%5C%5C%0A" />
<img src="https://tex.z-dn.net/?f=%28a%2Bb%29%5E3%20%3D%2015%5Csqrt%7B15%7D%5C%5C%0Aa%5E3%2Bb%5E3%2B3ab%28a%2Bb%29%3D%2015%5Csqrt%7B15%7D%5C%5C%0Aa%5E3%2Bb%5E3%2B%203%286%29%28%5Csqrt%7B15%7D%29%3D%2015%5Csqrt%7B15%7D%5C%5C%0Aa%5E3%2Bb%5E3%3D-3%5Csqrt%7B15%7D" />
entonces la respuesta es<img src="https://tex.z-dn.net/?f=-9%5Csqrt%7B15%7D" />.