<i>Copyright (c) Recommenders contributors.</i>

<i>Licensed under the MIT License.</i>

DKN : Deep Knowledge-Aware Network for News Recommendation

DKN [1] is a deep learning model which incorporates information from knowledge graph for better news recommendation. Specifically, DKN uses TransX [2] method for knowledge graph representaion learning, then applies a CNN framework, named KCNN, to combine entity embedding with word embedding and generate a final embedding vector for a news article. CTR prediction is made via an attention-based neural scorer.

Properties of DKN:

• DKN is a content-based deep model for CTR prediction rather than traditional ID-based collaborative filtering.
• It makes use of knowledge entities and common sense in news content via joint learning from semantic-level and knnowledge-level representations of news articles.
• DKN uses an attention module to dynamically calculate a user's aggregated historical representaition.

Data format:

DKN takes several files as input as follows:

• training / validation / test files: each line in these files represents one instance. Impressionid is used to evaluate performance within an impression session, so it is only used when evaluating, you can set it to 0 for training data. The format is :

[label] [userid] [CandidateNews]%[impressionid]

e.g., 1 train_U1 N1%0

• user history file: each line in this file represents a users' click history. You need to set his_size parameter in config file, which is the max number of user's click history we use. We will automatically keep the last his_size number of user click history, if user's click history is more than his_size, and we will automatically padding 0 if user's click history less than his_size. the format is :

[Userid] [newsid1,newsid2...]

e.g., train_U1 N1,N2

• document feature file: It contains the word and entity features of news. News article is represented by (aligned) title words and title entities. To take a quick example, a news title may be : Trump to deliver State of the Union address next week , then the title words value may be CandidateNews:34,45,334,23,12,987,3456,111,456,432 and the title entitie value may be: entity:45,0,0,0,0,0,0,0,0,0. Only the first value of entity vector is non-zero due to the word Trump. The title value and entity value is hashed from 1 to n(n is the number of distinct words or entities). Each feature length should be fixed at k(doc_size papameter), if the number of words in document is more than k, you should truncate the document to k words, and if the number of words in document is less than k, you should padding 0 to the end. the format is like:

[Newsid] [w1,w2,w3...wk] [e1,e2,e3...ek]

• word embedding/entity embedding/ context embedding files: These are npy files of pretrained embeddings. After loading, each file is a [n+1,k] two-dimensional matrix, n is the number of words(or entities) of their hash dictionary, k is dimension of the embedding, note that we keep embedding 0 for zero padding. In this experiment, we used GloVe[4] vectors to initialize the word embedding. We trained entity embedding using TransE[2] on knowledge graph and context embedding is the average of the entity's neighbors in the knowledge graph.

Global settings and imports

from recommenders.models.deeprec.deeprec_utils import *
from recommenders.models.deeprec.models.dkn import *
from recommenders.models.deeprec.io.dkn_iterator import *
import time

import tensorflow as tf
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)

data paths

Usually we will debug and search hyper-parameters on a small dataset. You can switch between the small dataset and full dataset by changing the value of tag.

tag = 'small' # small or full

data_path = 'data_folder/my/DKN-training-folder'

yaml_file = './dkn.yaml' #  os.path.join(data_path, r'../../../../../../dkn.yaml')
train_file = os.path.join(data_path, r'train_{0}.txt'.format(tag))
valid_file = os.path.join(data_path, r'valid_{0}.txt'.format(tag))
test_file = os.path.join(data_path, r'test_{0}.txt'.format(tag))
user_history_file = os.path.join(data_path, r'user_history_{0}.txt'.format(tag))
news_feature_file = os.path.join(data_path, r'../paper_feature.txt')
wordEmb_file = os.path.join(data_path, r'word_embedding.npy')
entityEmb_file = os.path.join(data_path, r'entity_embedding.npy')
contextEmb_file = os.path.join(data_path, r'context_embedding.npy')
infer_embedding_file = os.path.join(data_path, r'infer_embedding.txt')
    

Create hyper-parameters

epoch=5
hparams = prepare_hparams(yaml_file,
                          news_feature_file = news_feature_file,
                          user_history_file = user_history_file,
                          wordEmb_file=wordEmb_file,
                          entityEmb_file=entityEmb_file,
                          contextEmb_file=contextEmb_file,
                          epochs=epoch,
                          is_clip_norm=True,
                          max_grad_norm=0.5,
                          history_size=20,
                          MODEL_DIR=os.path.join(data_path, 'save_models'),
                          learning_rate=0.001,
                          embed_l2=0.0,
                          layer_l2=0.0,
                          use_entity=True,
                          use_context=True
                         )
print(hparams.values())

input_creator = DKNTextIterator

Train the DKN model

model = DKN(hparams, input_creator)

t01 = time.time()
print(model.run_eval(valid_file))
t02 = time.time()
print((t02-t01)/60)

model.fit(train_file, valid_file)

Now we can test again the performance on valid set:

t01 = time.time()
print(model.run_eval(test_file))
t02 = time.time()
print((t02-t01)/60)

Document embedding inference API

After training, you can get document embedding through this document embedding inference API. The input file format is same with document feature file. The output file fomrat is: [Newsid] [embedding]

model.run_get_embedding(news_feature_file, infer_embedding_file)

we compre with DKN performance between using knowledge entities or without using knowledge entities (DKN(-)):

Reference

[1] Wang, Hongwei, et al. "DKN: Deep Knowledge-Aware Network for News Recommendation." Proceedings of the 2018 World Wide Web Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 2018.