example_compare_amconll/system_2.amconll tmp/ example_compare_amconll/system_1.amconll. More detailed f-scores per dependency labels are printed to a file in the folder.Įxample call (assuming you have placed MaltEval.jar in analyzers): cd analyzersīash am_comparison_full.sh. It prints f-score and LAS/UAS, and will open the visualization tool. The bash script is documented/self-explanatory. Where file1 and file2 are the original amconll files you want to compare (this script calls the preparation python script for you) and folder is an empty, existing folder where the results will be put. Full default comparison + visualizationĬd to analyzers/ and call: bash am_comparison_full.sh file1 file2 folder/ There's also a paper with some usage examples from Nilsson and Nivre 2008. MaltEval is quite powerful, see here for a documentation. This View Mode will display the graph fragment and type below each word.
If you point your cursor on a word, a little label will pop up, telling you its graph fragment and type.Īlternatively, you can change the View Mode to Inline Scroll (default is 'Inline Zoom') in the settings. We can call MaltEval.jar to visuzalize the trees and explore in the errors as follows: java -jar analyzers/MaltEval.jar -g some/directory/gold_file.amconll -s some/directory/system_output.amconll -v 1 If we have placed the MaltEval.jar file inside the analyzers folder, This command intersects the two amconll files and sorts them by sentence length so you can look at short examples first. Where system_output.amconll are the parser outputs, gold_file.amconll contains gold AM dependency trees and some/directory/ directory will serve as storage of intermediate files (conll-like files that MaltEval can read). Internal note: Location on server where to find AMCoNLL files on the coli serversįirst, cd to analyzers/ and call: python prepare_visualize.py system_output.amconll gold_file.amconll some/directory/.Note would be useful to evaluate an AM dependency tree stored in an AMCoNLL file to the final graph: maybe have a look at evaluate the AM-CoNLL trees to graphs or here. See also Visualizing graphbanks for visualizing all graphs in a certain file (not only for MRP files) and the compare_mrp.py script. Prerequisite: appropriate MRP file exists. Get PDF with graph for one sentence: step by step description about how to visualize the graph itself (not its AM dependency tree).One AM dependency tree to LaTeX: am_dep_to_latex.py:īased on the sentence ID, produces the LaTeX and dot files for a specified AM dependency tree.Pro: don't need matching IDs, graph constants drawn using graphviz.Ĭon: no single framework view, not many options (less elaborate GUI compared to MaltEval), no F1 score computation or colorized diff. A much more simplistic version using ImageMagick instead of PyQt is also available. The own GUI approach: compare_amconll.py ( compare_amconll_qtgui.py):įor comparative (not single) framework analysis, using PyQt (and Pdflatex, Graphviz and Inkscape for drawing).Pro: F1 score computation and colorized diff, elaborate GUI.Ĭon: for comparative case need corpora with matching IDs per sentence, graph constants only in Penman notation. The MaltEval approach: am_comparison_full.sh and visualize_am_trees.sh:įor comparative or single framework analysis, using the MaltEval tool.When you want to perform error analysis on AM dependency trees, here are some scripts you can use (all in am-parser/analyzers) and some further remarks: Table of contents:
0 kommentar(er)
