Efficient text generation of user-defined topic using generative adversarial

networks

Chenhan Yuan Yi-chin Huang Cheng-Hung Tsai

Dept. of Computer Science Dept. of Computer Science Institute for Information Industry
Virginia Tech National Pingtung University Taipei, Taiwan
VA, USA Pingtung, Taiwan [email protected]
[email protected] [email protected]

Abstract rent neural network (RNN) (Mikolov et al., 2010)

to train the language model because of its capa-
This study focused on efficient text generation
bility to process sequential data. However, the
using generative adversarial networks (GAN).
arXiv:2006.12005v1 [cs.CL] 22 Jun 2020

Assuming that the goal is to generate a para- RNN suffers from the gradient vanishing problem
graph of a user-defined topic and sentimental (Hochreiter, 1998) when the sequence becomes
tendency, conventionally the whole network longer. To address this problem, Long Short-Term
has to be re-trained to obtain new results each Memory (LSTM) (Hochreiter and Schmidhuber,
time when a user changes the topic. This 1997)is further adopted as a sequential neural net-
would be time-consuming and impractical. work model to generate sentences.
Therefore, we propose a User-Defined GAN
Lately, the Generative Adversarial Networks
(UD-GAN) with two-level discriminators to
solve this problem. The first discriminator
(GAN) framework (Goodfellow et al., 2014) has
aims to guide the generator to learn paragraph- been introduced into the NLP community. GAN
level information and sentence syntactic struc- has two different models for completing the data-
ture, which is constructed by multiple-LSTMs. generating task. One of them is Generator G,
The second one copes with higher level in- which is responsible for generating data, and an-
formation, such as the user-defined sentiment other one is discriminator D, which determines
and topic for text generation. The cosine sim- whether the input data is the real data or not.
ilarity based on TF-IDF and length penalty
The generator G continuously optimizes generated
are adopted to determine the relevance of the
topic. Then, the second discriminator is re- data based on the judgment of discriminator D. Af-
trained with generator if the topic or senti- ter several epochs, the generated data will become
ment for text generation is modified. The more realistic.
system evaluations are conducted to compare However, GAN was originally designed to pro-
the performance of the proposed method with cess continuous data, and using discrete data as
other GAN-based ones. The objective results input would make it impossible to update the gra-
showed that the proposed method is capable
dients of the GAN framework(Huszár, 2015). To
of generating texts with less time than oth-
ers and the generated text are related to the process discrete data, several variants of the GAN
user-defined topic and sentiment. We will fur- model for generating text have been proposed.
ther investigate the possibility of incorporating These GAN variants could achieve good perfor-
more detailed paragraph information such as mances in text generation task, such as MaskGAN
semantics into text generation to enhance the (Fedus et al., 2018), RankGAN (Lin et al., 2017),
result. and TextGAN (Zhang et al., 2016).
In order to make these models fit the distribu-
1 Introduction
tion of real text data better, the number of pa-
Text generation, as a basic natural language pro- rameters of text generation models based on neu-
cessing task, has many applications, such as dia- ral network are increased, which means that train-
logue robots (Li et al., 2017), machine translation ing these neural network models often takes a lot
(Hu et al., 2017), paraphrasing (Power and Scott, of time even using GPU. Conventionally, topic-
2005) and so on. With the rise of deep learning, related text generation models incorporate an ar-
different neural networks are introduced to gener- bitrary topic as an input by adopting mechanisms
ate text. For example, researchers use the recur- like attention (Feng et al., 2018). Therefore, each
time when the user wants to generate new sen- First, we discussed the related works of our
tences with another topic or sentimental tendency, method in the section 2. The proposed method
the text generation models have to be retrained is described in the Section 3, including the fea-
with all parameters to satisfy new requirements. ture extraction and model definition and training.
In some scenarios, e.g., news generation, spend- In the Section 4, the experiment settings and eval-
ing lots of time retraining model is not practical uation results of the comparing methods are de-
and the user wants new responding quickly. picted. Finally, the concluding remarks and future
To tackle this problem, a novel text genera- works are described in the Section 5.
tion model based on GAN is proposed, which is
Algorithm 1 Initial training generator Gθ,
called User-Defined Generative Adversarial Net-
discriminator-special Dγ, discriminator-general
works (UD-GAN). The key idea is to separate the
Dφ
sentence syntax model as the basic model and the
topic-related model as a higher-level model, and 1: Initialize Gθ, Dφ and Dγ with random
these two could be trained independently from weightsθ, φ and γ
each other. So, when the topic or other user- 2: Pre-train Gθ using MLE on real text data set
defined information is modified, e.g., sentimental 3: Generate negative samples using Gθ to train
tendency, only one of both models needs to be re- Dφ and Dγ
trained. In this way, once the basic syntax model 4: Generate synthetic positive samples to train
is established, the following training will become Dγ
much faster, since only the higher-level model has 5: Minimizing the cross entropy to pre-train Dγ
to be retrained. 6: Minimizing the cross entropy to pre-trainDφ
7: for i ← 1 to M do
In our proposed method, the discriminator is
constructed based on this idea. One of the discrim- 8: for j ← 1 to N do
inators called discriminator-general, which learns 9: Generate a sequence Y1:T v Gθ
to determine the proper context information and 10: Compute rewards via Eq.5
whether the input sentence is a valid syntactic 11: Update parameters of Gθ via Eq.4
structure. Another discriminator is called the 12: end for
discriminator-special, which ensures the output is 13: for k ← 1 to P do
user-defined. Inspired by SeqGAN (Yu et al., 14: Generate negative samples using Gθ
2017), we use the evaluation results of the gener- 15: Train Dφ with negative samples and real
ated text from discriminators as a reward to guide text data via Eq.6
the generator to select future actions, which is to 16: end for
generate an updated word. 17: for l ← 1 to T do
18: Generate feature vectors corresponding
For training the discriminator-special, it will
to negative samples generated by Gθ
take feature vectors as input, instead of sentences.
19: Generate synthetic feature vectors
The feature vector is defined based on the senti-
20: Train Dγ with negative and synthetic
ment detection and topic relevance of generated
feature vectors via Eq.6
sentence. The cosine similarity based on TF-IDF
21: end for
and length penalty are jointly adopted to represent
22: end for
topic relevance.
Note that the UD-GAN is designed to be
more practical to generate short paragraphs,
2 Related Work
which means sentences generated by it should
be context-aware and behave like a paragraph to- Text generation is a basic task in natural language
gether with surrounding sentences. To achieve processing (NLP). In previous works, many re-
this idea, discriminator-general is designed with searchers (Power and Scott, 2005) extracted gram-
hierarchical multiple LSTM layers. The LSTM mar rules from text to generate new texts. These
at the top of the network processes paragraph- works are capable of generating semantically rich
level information while the bottom LSTMs pro- and grammatically correct text, but due to the fixed
cess sentence-level information. grammar rules, generated sentences are quite lack
The organization of the paper is as follows: of diversity. As neural networks could fit the dis-
 Figure 1: The framework of the proposed UD-GAN

Algorithm 2 Following training generator Gθ, uses this signal to guide the outputs in each gener-
discriminator-special Dγ ation step before generating the entire sentence.
1: Initialize Gθ, Dγ with random weightsθ, Dγ Another vital application of NLP is the senti-
2: Load trained Dφ ment analysis (Pang et al., 2008; Wilson et al.,
3: Do 2v5 steps in Algorithm 1 2005). Generally, the sentiment analysis task mea-
4: for i ← 1 to M do sures the emotional tendency of the whole sen-
5: Do 8v12 steps in Algorithm 1 tence based on the word usage that can represent
6: for l ← 1 to T do emotions in that sentence. Therefore, the estab-
7: Generate feature vectors corresponding lishment of an emotional word dictionary is essen-
to negative samples generated by Gθ tial. Affective Norms for English Words (ANEW)
8: Generate synthetic feature vectors (Bradley and Lang, 1999) lexicon sorts all words
9: Train Dγ with negative and synthetic according to rating score from 1 to 9. The highest
feature vectors via Eq.6 score means the sentence convey a very positive
10: end for emotion, and the lowest one represents the most
11: end for negative emotion for the sentence. Based on that,
some researchers (Hutto and Gilbert, 2014) con-
struct a gold-standard list of lexical features then
tribution of real data better, some researchers de- combine these lexical features with consideration
sign GAN-based models as language models to for five general rules, which could represent the
generate text. Unlike standard GAN, the loss func- sentiment of a sentence. The VADER algorithm
tion or training method of generator are modified proposes a rule-based sentiment analyzer that has
to enable GAN to process discrete data. outperformed the other machine learning-based al-
For example, In TextGAN (Zhang et al., 2016), gorithms.
researchers apply feature matching with standard
GAN loss function to train the generator. Rein- 3 Proposed Method
forcement learning (Sutton et al., 2000) is another
useful machine learning technique to train model 3.1 Basic Structure of UD-GAN
with unlabeled data. Trained model will choose As shown in Fig.1, UD-GAN contains a gen-
next actions to maximize expected reward, which erator Gθ that is capable of generating context-
is given by interface environment. Yu proposed dependent sentences and the two-level discrimina-
SeqGAN (Yu et al., 2017), which combine rein- tors. Discriminator-general Dφ guides the gener-
forcement learning with GAN. In SeqGAN, the ator to learn the paragraph-level information and
generator uses the result of discriminator as a re- correct syntactic structure, while discriminator-
ward and choose next actions, which is to generate special Dγ determines whether the generated text
the next words in text generation task. To gener- is related to the user-defined topic and senti-
ate longer text, LeakGAN (Guo et al., 2018) is in- ment. Discriminator-special Dγ is trained with
troduced to enable the discriminator leaks features synthetic perfect data and generated text data,
extracted from its input to generator, which then while discriminator-general Dφ is trained with real
text data and generated text data. defined by user is mapped in [0, 1] via Eq.1.
As we apply reinforcement learning with pol-
icy gradient to train the generator, the outputs penaltyg0 =
of the two discriminators for the generated text leng0 − leni (1)
will be combined and served as a reward to max |leng − leni | − min |leng − leni |
train the generator. Generator Gθ will choose g∈G g∈G
the best next actions based on the reward it re- where i is input sentence, g 0 is the evaluated
ceived. After the first training via Algorithm 1, generated sentence and G is the set of gener-
the discriminator-general parameters are saved as ated sentences. We set 0.5 to the optimal length
the pre-trained model. In the subsequent train- penalty, which means that if the length of the sen-
ings, we only train the parameters of the generator tence is very close to or very far from the length of
Gθ and discriminator-special Dγ via Algorithm 2. topic sentence, it is unqualified.
The details about training method and structure of We implemented the VADER algorithm to cal-
discriminators and generator are described as fol- culate the probability that a generated sentence be-
lows. longs to a positive, negative or neutral emotion
class. As VADER gives three values that corre-
spond to the probability of each sentiment cate-
3.2 The Framework of D-Special gory, the sum of which is 1, these three values will
be saved in the third to fifth elements. The optimal
sentiment is defined by the user.
The Feature Vector of D-Special
In conventional GAN training, the discrimina-
Discriminator-special Dγ takes a vector con- tor treats real text data as the positive sample and
taining 5 elements as input, which could represent generated text as the negative sample. However,
the sentimental and topical relevance of each sen- there is no sentence in real corpus that has ex-
tence. actly the same features as the positive sample,
since its feature vector is constructed by apply-
In our model, users can describe the cause and
ing the above mention algorithm, while the user-
effect of an event in one sentence, which is used as
defined feature vector is a specific value. There-
the topic for generating sentences. We use the first
fore, we train the discriminator-special Dγ with
element to represent the similarity between sen-
synthetic data, which is treated as positive sam-
tence entered by the user and generated sentence,
ple. For example, supposing that the user would
which could also represent the user-defined topic
like to generate an essay with one positive emo-
relevance of the generated text. Based on the TF-
tion, then the UD-GAN will generate [1,0.5,1,0,0]
IDF (Sparck Jones, 1972) value of each word in
vectors corresponding to the number of generated
the sentence that the user entered and the gener-
sentences, which will be combined with vectors
ated sentence, the cosine similarity between these
corresponding to the generated sentences as the in-
two sentences is calculated as a parameter to mea-
put of discriminator-special.
sure the user-defined topic relevance of the gener-
The Structure of D-Special
ated sentence. A larger value of cosine similarity
Two linear layers with Relu as the activation
means that the generated sentence is related to the
function are used as discriminator-special Dγ . The
user-defined topic.
output of this network will be part of the reward to
However, if only this element is used to in- train generator Gθ after it passed through a soft-
struct the generator Gθ to generate topic-related max layer.
sentences, the resulting sentences will be substan- We explain here why the multiple fully con-
tially as long as the user-defined topic sentence. nected layer is implemented as a discriminator-
More importantly, the generated sentences will special. The first reason is that after the
lack diversity with same meaning. Therefore, we Discriminator-General is constructed, in the sub-
propose the second element, length penalty, to sequent training, the discriminator-special will
reduce the negative impact of the first element. be continuously retrained when demands of user
The difference between the length of the gener- change. This requires spending as little time as
ated sentence and the length of the topic sentence possible to train a good discriminator-special. The
 Figure 2: The proposed framework for Discriminator-general

multiple fully connected layer has fewer param- 3.4 Generator

eters, which means this network will converge Generator Gθ is designed with GRU (Chung et al.,
faster than others will. Another reason is that the 2014). In UD-GAN, due to the excessive pa-
aim of training discriminator-special is to distin- rameters of the two discriminators, it is easy to
guish whether the input vector corresponds the guide the generator to be over-fitting. As a com-
user-defined one. For an input with only five monly used variant of LSTM, GRU avoids this
variables, a neural network with two fully con- over-fitting problem. In addition, having fewer pa-
nected layers is complicated enough to determine rameters than conventional LSTM allows GRU to
the class of input vector correctly. take less time to converge, which is the first prior-
ity in UD-GAN.
3.3 The Framework of D-General
3.5 Reward and Policy Gradient Training
Unlike conventional ideas of using classifier- The reinforcement learning has been incorporated
based models as a discriminator, the discriminator- to enable GAN to process discrete data. In this
general Dφ needs to process sequence data and scenario, generator Gθ will use the results from
context information, such as the paragraph infor- discriminators on the generated text as reward to
mation for each sentence to generate paragraph- generate next words. In UD-GAN, the reward is
level text. calculated based on results of two discriminators.
Therefore, as shown in Fig.2, we designed Generator Gθ tries to maximize expected reward
a hierarchical-multiple-LSTM neural network as from the initial state till the end state via Eq.3(loss
the discriminator-general Dφ . The bottom multi- function).
layers LSTM takes an embedding vector for each T
word in a sentence as the input and it outputs a J(θ) =
X
E(Rt |St−1 , θ)
feature matrix representing the corresponding sen- t=1
tence. The top bidirectional LSTM (Graves and T
X
Schmidhuber, 2005) takes the feature matrices of = Gθ (yt |Y )[λ(Dφ (Y )) + (1 − λ)Dγ (Y ))]
these sentences, which belong to the same para- t=1
graph, as input and it outputs a feature matrix (3)
representing that paragraph. After transforming
through two different linear layers respectively, Where λ is a manually set weight and Y is
the above two feature matrices will be combined a complete sequence and Rt is the reward for a
together. Finally, the discriminator-general calcu- whole sequence. In our experiments, we set λ
lates the score of the input sentence via Eq. 2. to 0.8 to give more weight to the discriminator-
general Dφ for generating sentences with better
syntactic structure. Note that since discrimina-
tors can only make the judgement with a complete
R(Y ) = sof tmax[(1 − β)LST Mα + βLST Mη ] sequence, the Monte Carlo search (Silver et al.,
(2) 2016) is adopted to find out some of the possible
where β is a trainable parameter ranging 0-1. generated complete sequences of each state. the
average judgment results of the discriminators for GAN-based models ROUGE-L
these sequences are calculated as a reward of this UD-GAN(GS) 364.73
state. UD-GAN(S) 370.54
In this paper, we implemented policy gradient UD-GAN(G) 340.19
method. The gradient of Eq.3 can be derived ap- SeqGAN 342.27
proximately as follows: LeakGAN 345.03

∇θ J(θ) ' Table 1: The ROUGE-L score for each system. UD-
T
GAN(G+S) represents initial training and UD-GAN(S)
X (4) represents following training. UD-GAN(G) only has
Eyt ∼Gθ [∇θ logGθ (yt |Y )QG
Dφ ,Dγ (yt |Y )]
θ
discriminator-general and generator. Note that this
t=1
score is the sum of all generated sentences’ ROUGE-
Gθ L results.
where QDφ ,Dγ
(yt |y1:t−1 ) can be derived via
Eq.5.
for the UD-GAN and the bottom LSTMs is
QGDφ ,Dγ (yt |y1:t−1 ) = λ(Dφ (Y ))+(1−λ)Dγ (Y ))
θ
set to 64. Besides, the hidden dimension of
(5)
discriminator-special linear layer and GRU unit
The loss function of both discriminators is intro-
of generator is set to 32. In each epoch of ini-
duced as follows:
tial training, generator G is trained once, and the
J = −(EY vPdata [R(Y )] − EY vGθ [1 − R(Y )]) discriminator-general is trained four times while
(6) the discriminator-special is trained twice.
where R(Y ) is the reward from two discriminators For evaluating the effectiveness of the proposed
for a whole sequence. method, we first compared the sentences relevance
to user-defined topic and sentimental tendency,
4 Experimental Analysis and then compare the training time of each sys-
tem. Finally, the fluency and correctness of UD-
4.1 Dataset GAN and baseliens were evaluated.
We crawled nearly 10,000 press released from the
opinion section of Newsweek as the training cor- 4.3 Relevance of Topic and Sentiment
pus. The opinion section of Newsweek is selected Relevance of Topic
as training corpus because the paragraphs of the As an objective summary accuracy evaluation
essays in Newsweek are generally closely related method that is widely used, ROUGE (Lin, 2004)
and not long. The other reason is that through the is also adopted here to evaluate whether gener-
articles in the opinion section, authors can often ated sentences are related to user-defined topics.
convey their own sentiment tendencies. Generated sentences are treated as summaries to
NER is used to replace name-entities with their be evaluated, and the topic sentence defined by
name-entity tags to decrease vocabulary. After to- user is used as a reference summary to evaluate
kenizing the corpus, long sentences of more than whether the generated sentence is related to the
45 words in the corpus were removed. The fi- topic. Note that even if the ROUGE scores of the
nal training corpus has 425K sentences and 103K generated sentences are not high, it does not mean
paragraphs. that these sentences are not closely related to the
user-defined topic necessarily. One possibility is
4.2 Experimental Setting that the generated sentences will use other words
SeqGAN and LeakGAN are used as the baseline or syntactic structures to describe the topic sen-
system to evaluate UD-GAN. We train SeqGAN tence.
and LeakGAN for 20 epochs, which is same as In this paper, we report the sum of ROUGE-L
the number of times UD-GAN is trained. Other scores of all sentences. Based on the longest com-
parameters of baselines remain unchanged as im- mon subsequence, ROUGE-L is a score related to
plemented in their original papers. recall rate. As shown in Table.1, the ROUGE-
The bottom of the discriminator-general con- L scores for UD-GAN (G+S) and UD-GAN(S)
sists of three layers of LSTM. The hidden dimen- are slightly higher than baseline systems and UD-
sion of discriminator-general bidirectional LSTMs GAN (G).
 Positive Negative Neutral
UD-GAN(GS) 0.39 0.05 0.56
topic: the attack in douma occurred days
UD-GAN(S) 0.41 0.04 0.55
after trump indicated that he wanted to pull
UD-GAN(G) 0.10 0.08 0.82
us troops out
SeqGAN 0.09 0.08 0.83
UD-GAN(S):
LeakGAN 0.08 0.07 0.85
1. the country contacts to the u.s. and trains
Table 2: The probability of sentiment tendency of gen- troops for government living on the federal
erated sentences system in LOCATION .
2. we are discussed actively : if u.s. is the facts
that citizens in the country will likely vote
Relevance of Sentimental Tendency for type elections ?
The VADER algorithm is used to calculate the 3. during these attack things occurred days , i
probability that the sentimental tendency of the say just PERSON who pulls in the exchange
generated sentences to be positive, negative or best troops out as trade in LOCATION .
neutral. Here, we evaluated the system perfor- 4. and he often enthusiastic , telling only having
mance by setting the target sentimental tendency heard nothing happened while you can indicate
as positive. to pull out from country .
As shown in Table.2, the average probability 5. but these generations in LOCATION can
in each sentimental tendency category of all sen- predict the next five attacks occur.
tences is calculated. With training discriminator- LeakGAN:
special, UD-GAN (G+S) and UD-GAN (S) are 1. it prompted the opposition during a “ real ”
more likely to generate positive sentences than of subtlety , and video straws .
baselines. Which proves that the proposed method 2. but if PERSON know that we serve the best
is capable to generate the sentences with the de- drives these country purposes . ”
sired sentiment. However, since the total number 3. besides disarming our administration and
of sentences expressing positive sentimental ten- pricing and its traditional views .
dency in the training corpus is quite low, the prob- 4. with her contempt for all enough neighbors . ”
ability of UD-GAN generating positive sentiment 5. one day i ’d go beyond my candor .
is still not higher than 0.5. SeqGAN:
Generate Context-dependent Sentences 1. we do n’t mean .
To demonstrate that UD-GAN can generate 2. you should be “ changed ” that you know .
context-dependent sentences, we show sentences 3. i ’ve always been proposing the findings .
generated by UD-GAN and baselines. As shown 4. in other words , he ’s because you have a
in Table 3, one can see that the proposed UD-GAN testament to his goodness – not a result .
does generate sentences related to the user-defined 5. he gave economic law .
topic. UD-GAN tries to add some conjunctions
Table 3: An example of the generated sentences from
when generating sentences so that the sentences
different systems
seem to be related, and each sentence is extended
with other related words based on the topic. Note
that there are some Name-Entity (NE) tags gen-
erated by the models because the NE tagging has
been done for simplifying the corpus lexicon. GAN-based models Time s
However, semantically, these sentences are not UD-GAN(GS) 29061.48
intrinsically related to each other, which is a prob- UD-GAN(S) 4841.99
lem we will address in the future. UD-GAN(G) 29036.65
SeqGAN 27011.08
4.4 Training Time Evaluation
LeakGAN 30471.95
The time spending on gradient propagation and
update of UD-GAN and baselines are compared, Table 4: Time spending on training of each models
instead of the time spending on loading and saving
data. Our platform is a workstation with a GeForce
GTX 1080 Ti graphics card with 11G RAM. GAN-based models BLEU score
All GAN-based models compared here are imple- UD-GAN(G+S) 0.6412
mented in pytorch (Paszke et al., 2017) framework UD-GAN(S) 0.6409
to eliminate the impact of different frameworks on UD-GAN(G) 0.6357
time consumption. SeqGAN 0.6303
LeakGAN 0.7161
As shown in Table.4, because the structure of
discriminator-general is more complex than the Table 5: The average BLEU score for each system.
structure of discriminator D of baselines, ini- Note that UD-GAN(S) achieves comparable BLEU
tial training of UD-GAN takes the longest time. performance with baselines, whose training needs far
However, in the subsquent trainings, due to the less time than baselines.
gradient propagation and parameter update of
discriminator-special is quite fast, the time re-
quired to train UD-GAN (S) is the shortest. The 5 Conclusion and Future Work
UD-GAN (S) takes only about an hour and a half In this paper, we propose a UD-GAN method to
to complete training, which is much less than the re-train text generation model more efficiently to
nearly eight hours of training time for baselines. generate sentences that are consistent with the new
user-defined topic and sentimental tendency. We
compared the accuracy and fluency of sentences
4.5 Fluency and Accuracy generated by UD-GAN with other GAN-based
text generation models. The experimental results
As shown in table 5, we report BLEU (Papineni showed that sentences generated by UD-GAN are
et al., 2002) scores of UD-GAN and baselines competent. Meanwhile, UD-GAN takes much
to compare the fluency and accuracy of text they less time in the re-train stage than other models.
generate. The BLEU we use here is the average According to experimental results, UD-GAN can
value of 1-gram BLEU, 2-gram BLEU and 3-gram also successfully generate sentences related to the
BLEU, which are given the same weights . user-defined topic and sentimental tendency, while
In the case of training the discriminator-general baselines does not have this capability. Besides,
only, the BLEU score of the UD-GAN (G) is be- UD-GAN can also generate paragraph-level text.
tween SeqGAN and LeakGAN. Therefore, the ac- However, the sentences generated by UD-GAN
curacy and fluency evaluation of using multi-layer are still inferior to the state-of-the-art method, i.e.,
LSTMs as a discriminator is comparable to that LeakGAN, in terms of fluency. And the current
of using a classifier-based model, such as CNN, paragraph-level information used here does not in-
as the discriminator. When the discriminator- clude complex linguistic information, such as the
general and discriminator-special are simultane- order of sentences. In future work, we will try
ously trained (initial training), UD-GAN (G+S) to maintain the existing advantages of UD-GAN
has a slightly higher BLEU score than UD-GAN while improving the readability of generated text.
(G). That is to say, even if discriminator-special
is added and the result of discriminator-general, References
which can distinguish the correctness of the sen-
Margaret M Bradley and Peter J Lang. 1999. Affective
tence, is less weighted, the resultant generator norms for english words (anew): Instruction manual
of UD-GAN (G+S) can still learn how to gen- and affective ratings. Technical report, Citeseer.
erate a sentence with the correct syntax. Then
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho,
we change the user-defined topic and sentimen- and Yoshua Bengio. 2014. Empirical evaluation of
tal tendency to train the discriminator-special only gated recurrent neural networks on sequence model-
(subsequent training). The results showed that the ing. arXiv preprint arXiv:1412.3555.
BLEU score of the UD-GAN(S) is still between William Fedus, Ian Goodfellow, and Andrew M Dai.
LeakGAN and SeqGAN. It means that retraining 2018. Maskgan: Better text generation via filling in
the discriminator-special has no effect on whether the . arXiv preprint arXiv:1801.07736.
the generator can learn the correct syntax with- Xiaocheng Feng, Ming Liu, Jiahao Liu, Bing Qin, Yibo
out changing the weights of rewards generated by Sun, and Ting Liu. 2018. Topic-to-essay generation
discriminator-general and discriminator-special. with neural networks. In IJCAI, pages 4078–4084.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Kishore Papineni, Salim Roukos, Todd Ward, and Wei-
Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Jing Zhu. 2002. Bleu: a method for automatic eval-
Courville, and Yoshua Bengio. 2014. Generative ad- uation of machine translation. In Proceedings of
versarial nets. In Advances in neural information the 40th annual meeting on association for compu-
processing systems, pages 2672–2680. tational linguistics, pages 311–318. Association for
Computational Linguistics.
Alex Graves and Jürgen Schmidhuber. 2005. Frame-
wise phoneme classification with bidirectional lstm Adam Paszke, Sam Gross, Soumith Chintala, Gre-
and other neural network architectures. Neural Net- gory Chanan, Edward Yang, Zachary DeVito, Zem-
works, 18(5-6):602–610. ing Lin, Alban Desmaison, Luca Antiga, and Adam
Lerer. 2017. Automatic differentiation in pytorch.
Jiaxian Guo, Sidi Lu, Han Cai, Weinan Zhang, Yong
Yu, and Jun Wang. 2018. Long text generation Richard Power and Donia Scott. 2005. Automatic gen-
via adversarial training with leaked information. In eration of large-scale paraphrases.
Thirty-Second AAAI Conference on Artificial Intelli-
David Silver, Aja Huang, Chris J Maddison, Arthur
gence.
Guez, Laurent Sifre, George Van Den Driessche, Ju-
lian Schrittwieser, Ioannis Antonoglou, Veda Pan-
Sepp Hochreiter. 1998. The vanishing gradient prob-
neershelvam, Marc Lanctot, et al. 2016. Mastering
lem during learning recurrent neural nets and prob-
the game of go with deep neural networks and tree
lem solutions. International Journal of Uncer-
search. nature, 529(7587):484.
tainty, Fuzziness and Knowledge-Based Systems,
6(02):107–116. Karen Sparck Jones. 1972. A statistical interpretation
of term specificity and its application in retrieval.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Journal of documentation, 28(1):11–21.
Long short-term memory. Neural computation,
9(8):1735–1780. Richard S Sutton, David A McAllester, Satinder P
Singh, and Yishay Mansour. 2000. Policy gradi-
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan ent methods for reinforcement learning with func-
Salakhutdinov, and Eric P Xing. 2017. Controllable tion approximation. In Advances in neural informa-
text generation. arXiv preprint arXiv:1703.00955, tion processing systems, pages 1057–1063.
7.
Theresa Wilson, Janyce Wiebe, and Paul Hoffmann.
Ferenc Huszár. 2015. How (not) to train your genera- 2005. Recognizing contextual polarity in phrase-
tive model: Scheduled sampling, likelihood, adver- level sentiment analysis. In Proceedings of Human
sary? arXiv preprint arXiv:1511.05101. Language Technology Conference and Conference
on Empirical Methods in Natural Language Pro-
Clayton J Hutto and Eric Gilbert. 2014. Vader: A par- cessing.
simonious rule-based model for sentiment analysis
of social media text. In Eighth international AAAI Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu.
conference on weblogs and social media. 2017. Seqgan: Sequence generative adversarial nets
with policy gradient. In Thirty-First AAAI Confer-
Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, ence on Artificial Intelligence.
Alan Ritter, and Dan Jurafsky. 2017. Adversar-
ial learning for neural dialogue generation. arXiv Yizhe Zhang, Zhe Gan, and Lawrence Carin. 2016.
preprint arXiv:1701.06547. Generating text via adversarial training. In NIPS
workshop on Adversarial Training, volume 21.
Chin-Yew Lin. 2004. Rouge: A package for auto-
matic evaluation of summaries. Text Summarization
Branches Out.

Kevin Lin, Dianqi Li, Xiaodong He, Zhengyou Zhang,

and Ming-Ting Sun. 2017. Adversarial ranking for
language generation. In Advances in Neural Infor-
mation Processing Systems, pages 3155–3165.

Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan

Černockỳ, and Sanjeev Khudanpur. 2010. Recurrent
neural network based language model. In Eleventh
annual conference of the international speech com-
munication association.

Bo Pang, Lillian Lee, et al. 2008. Opinion mining and

sentiment analysis. Foundations and Trends R in In-
formation Retrieval, 2(1–2):1–135.