Pressbooks has two options for LaTeX rendering in your book:
Note: LaTeX will not render in the visual editor, only the public facing side of your book. To see how it looks, use the "Preview" button after saving your work.
Since H5P is a plugin and not a native feature of Pressbooks, it is not impacted by any custom preamble you create for QuickLaTeX.
H5P uses MathJax to render LaTeX, so you can use the MathJax demo page to test if the code will appear as you expect.
If an equation must appear a certain way that MathJax cannot handle, you can present the question as an image rather than text in H5P, although this is not ideal for accessibility.
LaTeX cannot render in chapter headers or titles. To insert special characters, you can use the "Special Characters" option in the editing toolbar. The character can then be copied and pasted into the title.
Alternately, you can use an HTML/XML shortcode for the character. You can use either the Numeric Character Reference or the Character Entity Reference. Character references should appear like so:&reference; For example, to make an ampersand (&) appear in the title, you would enter one of the following: