Lời giải:
a) \(f'(x) = x^2 + (2 - m)x - (m + 3) \geq 0\), \(\forall x \in (0, +\infty)\)
\(\Leftrightarrow x^2 + 2x - 3 \geq m(x + 1), \forall x \in (0, +\infty)\)
\(\Leftrightarrow \frac{x^2 + 2x - 3}{x + 1} \geq m, \forall x \in (0, +\infty)\)
Đặt \(g(x) = \frac{x^2 + 2x - 3}{x + 1}\) \(\Rightarrow g'(x) = \frac{x^2 + 2x + 5}{(x + 1)^2} > 0, \forall x \neq -1\)
\(g(x) \geq m, \forall x \in (0, +\infty) \Leftrightarrow m \leq -3\). Chọn đáp án \(\boxed{A}\).
b) \(f'(x) \leq 0\), \(\forall x \in (0, 1)\)
\(\Leftrightarrow f'(x) = 0\) có 2 nghiệm \(x_1, x_2\) thỏa mãn \(x_1 \leq 0 < 1 \leq x_2\)
\(\Leftrightarrow \begin{cases} af'(0) \leq 0 \\ af'(1) \leq 0 \end{cases} \)
\(\Leftrightarrow \begin{cases} -(m+3) \leq 0 \\ -2m \leq 0 \end{cases} \)
\(\Leftrightarrow \begin{cases} m \geq -3 \\ m \geq 0 \end{cases} \iff m \geq 0 \)
Cách 2:
\(x^2 + (2 - m)x - (m + 3) \leq 0\), \(\forall x \in (0, 1)\)
\(\Leftrightarrow x^2 + 2x - 3 \leq m(x+1) \)
\(\Leftrightarrow \frac{x^2 + 2x - 3}{x + 1} \leq m\), \(\forall x \in (0, 1)\)
\(\Leftrightarrow m \geq 0 \)