The less of the smaller-sized duct you have, the better - so run 4" as far as possible before reducing to 3" for the wall penetration - that reduces the extent of the friction loss from smaller duct, and thus reduces the performance lost.
It won't kill the motor. It will move less air than it would with 4" duct the whole way.
Since this misconception that restricting fans puts more load on the motor comes up rather frequently, I made some tests.
For a "mixed flow axial" fan (radon fan, specifically) unrestricted 4" pipe 18W, restricted down to 2" 16W.
For a "squirrel cage blower" 29-30W unrestricted, 24W restricted.
For 3 different "axial" (propellor-type) fans, power did not change at all with restriction - neither more nor less. While that might negatively affect a marginal design due to inadequate cooling with a severely restricted airflow, a mere 4"->3" reduction would be unlikely to have that effect, and many bathroom fans are, in fact, one of the other types I tested. The idea that there's "more load (or strain) on the motor" is, however, false, as measured by the power consumption of the motor.
Less mass (and volume) of air is moved, and in physics terms that means less work (physics definition of work) is done. A hypothesis I have not tested is that the axial fans are likely moving the same mass of air because it's circulating around the tips of the fan blades when a restriction is applied, so they are moving it in circles (and will also be heating it up.)
Right - I'll have to remember to test a vacuum, as well. Not that it's terribly applicable to this question.