Ideas required to solve the problem:

The general solution of any trigonometric equation is given as –

• sin x = sin y, implies x = nπ + (– 1)ⁿy, where n ∈ Z.

• cos x = cos y, implies x = 2nπ ± y, where n ∈ Z.

• tan x = tan y, implies x = nπ + y, where n ∈ Z.

Given,

3sin² x – 5 sin x cos x + 8 cos² x = 2

⇒ 3sin² x + 3 cos² x – 5sin x cos x + 5 cos² x = 2

⇒ 3 - 5sin x cos x + 5 cos² x = 2 {∵ sin² x + cos ²x = 1 }

⇒ 5cos² x + 1 = 5sin x cos x

Squaring both sides:

⇒ (5cos² x + 1)² = (5sin x cos x)²

⇒ 25cos⁴ x + 10cos² x + 1 = 25 sin² x cos² x

⇒ 25cos⁴ x + 10cos² x + 1 = 25 (1 - cos² x) cos² x

⇒ 50cos⁴ x – 15 cos² x + 1 = 0

⇒ 50cos⁴ x – 10 cos² x – 5cos² x + 1 = 0

⇒ 10cos² x ( 5cos² x – 1) – (5cos² x – 1) = 0

⇒ (10cos² x - 1)(5cos² x – 1) = 0

∴ cos² x = 1/10 or cos² x = 1/5

Hence, when cos² x = 1/10

We have,

If cos x = cos y, implies x = 2nπ ± y, where n ∈ Z.

let cos α = 1/√10

∴ cos (π – α) = -1/√10

∴ x = 2nπ ± α or x = 2nπ ± (π – α)

∴ when,

When cos² x = 1/5

We have, .

If cos x = cos y, implies x = 2mπ ± y, where n ∈ Z.

let cos β = 1/√5

∴ cos (π – β) = -1/√5

∴ x = 2mπ ± β or x = 2mπ ± (π – β)

∴ when, .

…ans