Subjective, multi-attribute choice decisions – such as whom to marry or which college to attend – play a substantial role in decision makers’ long-term well-being. However, the metacognition literature lacks tools for assessing metacognitive capacities in subjective decisions. We present 3 studies in which we propose and validate the Knowledge of Weights (KoW) paradigm, a novel method for assessing metacognitive knowledge of attribute weights in subjective, multi-attribute choice decisions. In Study 1, we demonstrate the test-retest reliability of metrics generated by the KoW paradigm. In Study 2, we apply the KoW paradigm in four similar domains and show that it generates consistent results across the domains. In Study 3, we demonstrate that participants who perform better on the KoW paradigm make choices with which they are more satisfied, establishing predictive validity. These results suggest that the KoW paradigm is psychometrically sound. Use cases in cognitive psychology and beyond are discussed.