2026-07-22T22:18:06Z https://drops.dagstuhl.de/oai

oai:drops-oai.dagstuhl.de:6009 2024-03-06T10:29:53Z ddc:004 open_access

Comparing and Benchmarking Semantic Measures Using SMComp Costa, Teresa Leal, José Paulo Semantic similarity semantic relatedness testbed web application The goal of the semantic measures is to compare pairs of concepts, words, sentences or named entities. Their categorization depends on what they measure. If a measure only considers taxonomy relationships is a similarity measure; if it considers all type of relationships it is a relatedness measure. The evaluation process of these measures usually relies on semantic gold standards. These datasets, with several pairs of words with a rating assigned by persons, are used to assess how well a semantic measure performs. There are a few frameworks that provide tools to compute and analyze several well-known measures. This paper presents a novel tool - SMComp - a testbed designed for path-based semantic measures. At its current state, it is a domain-specific tool using three different versions of WordNet. SMComp has two views: one to compute semantic measures of a pair of words and another to assess a semantic measure using a dataset. On the first view, it offers several measures described in the literature as well as the possibility of creating a new measure, by introducing Java code snippets on the GUI. The other view offers a large set of semantic benchmarks to use in the assessment process. It also offers the possibility of uploading a custom dataset to be used in the assessment. Schloss Dagstuhl – Leibniz-Zentrum für Informatik Teresa Costa and José Paulo Leal 2016 Is Part Of OASIcs, Volume 51, 5th Symposium on Languages, Applications and Technologies (SLATE'16) (2016) InProceedings Text doc-type:ResearchArticle publishedVersion application/pdf doi:10.4230/OASIcs.SLATE.2016.4 urn:nbn:de:0030-drops-60090 https://drops.dagstuhl.de/entities/document/10.4230/OASIcs.SLATE.2016.4 eng https://creativecommons.org/licenses/by/3.0/legalcode