textproc/py-sentencepiece - The NetBSD Packages Collection

Unsupervised text tokenizer for Neural Network-based text generation 

SentencePiece is an unsupervised text tokenizer and detokenizer
mainly for Neural Network-based text generation systems where the
vocabulary size is predetermined prior to the neural model training.
SentencePiece implements subword units (e.g., byte-pair-encoding
(BPE)) and unigram language model with the extension of direct
training from raw sentences. SentencePiece allows us to make a
purely end-to-end system that does not depend on language-specific
pre/postprocessing.

This package contains the Python module.

Build dependencies

devel/py-setuptools pkgtools/mktools devel/pkgconf pkgtools/cwrappers

Runtime dependencies

textproc/sentencepiece lang/python311 lang/python311

Binary packages

OSArchitectureVersion
NetBSD 10.0aarch64py310-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py311-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py312-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py38-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py39-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py310-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py311-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py38-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py39-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py310-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py311-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py312-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py38-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64py39-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64ebpy310-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64ebpy311-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64ebpy312-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64ebpy38-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64ebpy39-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64ebpy310-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64ebpy311-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64ebpy312-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64ebpy38-sentencepiece-0.1.97.tgz
NetBSD 10.0aarch64ebpy39-sentencepiece-0.1.97.tgz
NetBSD 10.0alphapy310-sentencepiece-0.1.97.tgz
NetBSD 10.0alphapy311-sentencepiece-0.1.97.tgz
NetBSD 10.0alphapy312-sentencepiece-0.1.97.tgz
NetBSD 10.0alphapy38-sentencepiece-0.1.97.tgz
NetBSD 10.0alphapy39-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy310-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy311-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy312-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy38-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy39-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy310-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy311-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy38-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy39-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy310-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy311-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy312-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy38-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv6hfpy39-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy310-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy311-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy312-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy38-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy39-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy310-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy311-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy38-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy39-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy310-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy311-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy312-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy38-sentencepiece-0.1.97.tgz
NetBSD 10.0earmv7hfpy39-sentencepiece-0.1.97.tgz
NetBSD 10.0i386py310-sentencepiece-0.1.97.tgz
NetBSD 10.0i386py311-sentencepiece-0.1.97.tgz
NetBSD 10.0i386py312-sentencepiece-0.1.97.tgz
NetBSD 10.0i386py38-sentencepiece-0.1.97.tgz
NetBSD 10.0i386py39-sentencepiece-0.1.97.tgz
NetBSD 10.0i386py310-sentencepiece-0.1.97.tgz
NetBSD 10.0i386py311-sentencepiece-0.1.97.tgz
NetBSD 10.0i386py312-sentencepiece-0.1.97.tgz
NetBSD 10.0i386py38-sentencepiece-0.1.97.tgz
NetBSD 10.0i386py39-sentencepiece-0.1.97.tgz
NetBSD 10.0powerpcpy310-sentencepiece-0.1.97.tgz
NetBSD 10.0powerpcpy311-sentencepiece-0.1.97.tgz
NetBSD 10.0powerpcpy38-sentencepiece-0.1.97.tgz
NetBSD 10.0powerpcpy39-sentencepiece-0.1.97.tgz
NetBSD 10.0x86_64py310-sentencepiece-0.1.97.tgz
NetBSD 10.0x86_64py311-sentencepiece-0.1.97.tgz
NetBSD 10.0x86_64py312-sentencepiece-0.1.97.tgz
NetBSD 10.0x86_64py38-sentencepiece-0.1.97.tgz
NetBSD 10.0x86_64py39-sentencepiece-0.1.97.tgz
NetBSD 10.0x86_64py310-sentencepiece-0.1.97.tgz
NetBSD 10.0x86_64py311-sentencepiece-0.1.97.tgz
NetBSD 10.0x86_64py312-sentencepiece-0.1.97.tgz
NetBSD 10.0x86_64py38-sentencepiece-0.1.97.tgz
NetBSD 10.0x86_64py39-sentencepiece-0.1.97.tgz
NetBSD 8.0powerpcpy310-sentencepiece-0.1.97.tgz
NetBSD 8.0powerpcpy311-sentencepiece-0.1.97.tgz
NetBSD 8.0powerpcpy38-sentencepiece-0.1.97.tgz
NetBSD 8.0powerpcpy39-sentencepiece-0.1.97.tgz
NetBSD 8.0powerpcpy310-sentencepiece-0.1.97.tgz
NetBSD 8.0powerpcpy311-sentencepiece-0.1.97.tgz
NetBSD 8.0powerpcpy312-sentencepiece-0.1.97.tgz
NetBSD 8.0powerpcpy38-sentencepiece-0.1.97.tgz
NetBSD 8.0powerpcpy39-sentencepiece-0.1.97.tgz
NetBSD 8.0x86_64py310-sentencepiece-0.1.97.tgz
NetBSD 8.0x86_64py311-sentencepiece-0.1.97.tgz
NetBSD 8.0x86_64py38-sentencepiece-0.1.97.tgz
NetBSD 8.0x86_64py39-sentencepiece-0.1.97.tgz
NetBSD 8.0x86_64py310-sentencepiece-0.1.97.tgz
NetBSD 8.0x86_64py311-sentencepiece-0.1.97.tgz
NetBSD 8.0x86_64py312-sentencepiece-0.1.97.tgz
NetBSD 8.0x86_64py38-sentencepiece-0.1.97.tgz
NetBSD 8.0x86_64py39-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py310-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py311-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py312-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py38-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py39-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py310-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py311-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py38-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py39-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py310-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py311-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py312-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py38-sentencepiece-0.1.97.tgz
NetBSD 9.0aarch64py39-sentencepiece-0.1.97.tgz
NetBSD 9.0alphapy310-sentencepiece-0.1.97.tgz
NetBSD 9.0alphapy311-sentencepiece-0.1.97.tgz
NetBSD 9.0alphapy38-sentencepiece-0.1.97.tgz
NetBSD 9.0alphapy39-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy310-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy311-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy38-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy39-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy310-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy311-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy38-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy39-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy310-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy311-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy38-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv6hfpy39-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy310-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy311-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy38-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy39-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy310-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy311-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy38-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy39-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy310-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy311-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy38-sentencepiece-0.1.97.tgz
NetBSD 9.0earmv7hfpy39-sentencepiece-0.1.97.tgz
NetBSD 9.0i386py310-sentencepiece-0.1.97.tgz
NetBSD 9.0i386py311-sentencepiece-0.1.97.tgz
NetBSD 9.0i386py38-sentencepiece-0.1.97.tgz
NetBSD 9.0i386py39-sentencepiece-0.1.97.tgz
NetBSD 9.0i386py310-sentencepiece-0.1.97.tgz
NetBSD 9.0i386py311-sentencepiece-0.1.97.tgz
NetBSD 9.0i386py38-sentencepiece-0.1.97.tgz
NetBSD 9.0i386py39-sentencepiece-0.1.97.tgz
NetBSD 9.0powerpcpy310-sentencepiece-0.1.97.tgz
NetBSD 9.0powerpcpy311-sentencepiece-0.1.97.tgz
NetBSD 9.0powerpcpy312-sentencepiece-0.1.97.tgz
NetBSD 9.0powerpcpy38-sentencepiece-0.1.97.tgz
NetBSD 9.0powerpcpy39-sentencepiece-0.1.97.tgz
NetBSD 9.0powerpcpy310-sentencepiece-0.1.97.tgz
NetBSD 9.0powerpcpy311-sentencepiece-0.1.97.tgz
NetBSD 9.0powerpcpy38-sentencepiece-0.1.97.tgz
NetBSD 9.0powerpcpy39-sentencepiece-0.1.97.tgz
NetBSD 9.0sparc64py310-sentencepiece-0.1.97.tgz
NetBSD 9.0sparc64py311-sentencepiece-0.1.97.tgz
NetBSD 9.0sparc64py38-sentencepiece-0.1.97.tgz
NetBSD 9.0sparc64py39-sentencepiece-0.1.97.tgz
NetBSD 9.0x86_64py310-sentencepiece-0.1.97.tgz
NetBSD 9.0x86_64py311-sentencepiece-0.1.97.tgz
NetBSD 9.0x86_64py312-sentencepiece-0.1.97.tgz
NetBSD 9.0x86_64py38-sentencepiece-0.1.97.tgz
NetBSD 9.0x86_64py39-sentencepiece-0.1.97.tgz
NetBSD 9.0x86_64py310-sentencepiece-0.1.97.tgz
NetBSD 9.0x86_64py311-sentencepiece-0.1.97.tgz
NetBSD 9.0x86_64py312-sentencepiece-0.1.97.tgz
NetBSD 9.0x86_64py38-sentencepiece-0.1.97.tgz
NetBSD 9.0x86_64py39-sentencepiece-0.1.97.tgz
NetBSD 9.3x86_64py310-sentencepiece-0.1.97.tgz
NetBSD 9.3x86_64py311-sentencepiece-0.1.97.tgz
NetBSD 9.3x86_64py312-sentencepiece-0.1.97.tgz
NetBSD 9.3x86_64py38-sentencepiece-0.1.97.tgz
NetBSD 9.3x86_64py39-sentencepiece-0.1.97.tgz

Binary packages can be installed with the high-level tool pkgin (which can be installed with pkg_add) or pkg_add(1) (installed by default). The NetBSD packages collection is also designed to permit easy installation from source.

Available build options

(none)

Known vulnerabilities

The pkg_admin audit command locates any installed package which has been mentioned in security advisories as having vulnerabilities.

Please note the vulnerabilities database might not be fully accurate, and not every bug is exploitable with every configuration.

Problem reports, updates or suggestions for this package should be reported with send-pr.