3
是否有适用于python的好的即用型语言模型?
我正在为一个应用程序制作原型,我需要一个语言模型来计算一些生成的句子的困惑度。 我可以随时使用经过训练的python语言模型吗?简单的东西 model = LanguageModel('en') p1 = model.perplexity('This is a well constructed sentence') p2 = model.perplexity('Bunny lamp robert junior pancake') assert p1 < p2 我看过一些框架,但找不到我想要的。我知道我可以使用类似: from nltk.model.ngram import NgramModel lm = NgramModel(3, brown.words(categories='news')) 这在Brown Corpus上使用了很好的图林概率分布,但是我正在一些大型数据集(例如1b单词数据集)上寻找精心设计的模型。我可以真正相信一般领域的结果(不仅是新闻)
11
python
nlp
language-model
r
statistics
linear-regression
machine-learning
classification
random-forest
xgboost
python
sampling
data-mining
orange
predictive-modeling
recommender-system
statistics
dimensionality-reduction
pca
machine-learning
python
deep-learning
keras
reinforcement-learning
neural-network
image-classification
r
dplyr
deep-learning
keras
tensorflow
lstm
dropout
machine-learning
sampling
categorical-data
data-imputation
machine-learning
deep-learning
machine-learning-model
dropout
deep-network
pandas
data-cleaning
data-science-model
aggregation
python
neural-network
reinforcement-learning
policy-gradients
r
dataframe
dataset
statistics
prediction
forecasting
r
k-means
python
scikit-learn
labels
python
orange
cloud-computing
machine-learning
neural-network
deep-learning
rnn
recurrent-neural-net
logistic-regression
missing-data
deep-learning
autoencoder
apache-hadoop
time-series
data
preprocessing
classification
predictive-modeling
time-series
machine-learning
python
feature-selection
autoencoder
deep-learning
keras
tensorflow
lstm
word-embeddings
predictive-modeling
prediction
machine-learning-model
machine-learning
classification
binary
theory
machine-learning
neural-network
time-series
lstm
rnn
neural-network
deep-learning
keras
tensorflow
convnet
computer-vision