Psalm 2:11

Serve the Lord with fear
Not the creature, neither more, nor besides, nor with the Creator; God and mammon cannot both be served; nor any fictitious and nominal deities, the idols of the Gentiles, who are not gods by nature; but the true Jehovah, the one and only Lord God, he only is to be worshipped and served, even Father, Son, and Spirit. Here it may be understood either of the Lord Christ, the Son of God, who is to be served by the kings and judges of the earth, he being King of kings, and Lord of lords; or rather of Jehovah the Father, since the Son seems to be distinguished from him in ( Psalms 2:12 ) : and the service these persons are called unto lies not in the discharge of any office in the church, as in preaching the word, which is serving God in the Gospel of his Son; and hence the ministers of the word are eminently called the servants of the most high God; for kings and judges are not required hereby to lay aside their crowns and sceptres, and leave their seats of justice, and become preachers of the Gospel; but in acting according to the will of God revealed in his word, and in the whole worship of him, both internal and external: and this is to be done "with fear", not with fear of man, nor with servile fear of God, but with a godly and filial fear, with a reverential affection for him, and in a way agreeable to his mind and will; with reverence and awe of him, without levity, carelessness, and negligence;

and rejoice with trembling;
some reference may be had to the joy in public worship, as at sacrifices and festivals, and the music in divine service under the law; and the singing of psalms and hymns and spiritual songs under the Gospel; and especially to the Gospel dispensation itself, which is a time of joy and rejoicing; the Gospel is good tidings of great joy; the kingdom of God is not in things external, but in joy in the Holy Ghost; and, above all, respect is had to a rejoicing in Christ Jesus, in his person, righteousness, and salvation: and which is consistent with "trembling"; not with a fearful looking for of judgment, but with modesty and humility; in which sense this word, when joined with "fear" as here, is used ( Philippians 2:12 ) , and stands opposed to pride, haughtiness, and arrogance; men should so rejoice in Christ as to have no confidence in the flesh, or assume any degree of glory to themselves, or have any rejoicing in themselves, but wholly in Christ, giving all the glory of what they have to him.