Evaluation Abstract Meaning Representation (AMR)
Evaluation of future AMR parsers, and
Studies of inter-annotator agreement
we developed the
metric for comparing one AMR against another. It computes an f-score over the semantic triples inside AMRs.
. "Smatch: an Evaluation Metric for Semantic Feature Structures," (S. Cai and K. Knight), Proc. ACL, 2013.
Download Smatch code:
Evaluate AMRs with Smatch
quick Smatch tutorial
Smatch guide (pdf)