This article presents the results we obtained during BioASQ Task 12B Phase A on document ranking. Our strategy is based on a two-stage retrieval approach composed of a retriever and a reranker. The retriever is based on BM25 scoring and RM3 query expansion. The ranker is a BERT cross-encoder pre-trained on a biomedical corpus (BioLinkBERT). We study the impact of incorporating domain knowledge (MeSH) into this Pretrained Language Model and build a voting system to combine the insights from multiple models. Independently from the challenge, we also investigate a way to reduce the number of input tokens to bypass BERT limitation of 512 tokens for the input sequence.