Nlp
Use CNN for text tasks: embedding layer, convolution and pooling layers.
What is “MaxPooling” in CNN?
- Max-pooling is a sample-based discretization process
- Global max pooling = ordinary max pooling layer with pool size equals to the size of the input (minus filter size + 1, to be precise). You can see that MaxPooling1D takes a pool_length argument, whereas GlobalMaxPooling1D does not.
RNN is typically less parallel thus taking a longer time than CNN.
https://stats.stackexchange.com/questions/153531/what-is-batch-size-in-neural-network
Writing code for NLP research
Copy first for fast prototyping.
KEEP TRACK when you are doing experiments.
Use TENSORBOARD.
Designing components
Docker. file_cache.cached_path
conda create -n your-env-name python=3.6
Error analysis: https://nlpers.blogspot.com/2007/01/error-analysis.html
- When multiple techniques exist for solving the same problem, and they get reasonably close scores, is this because they are making the same sort of errors or different sorts?
- If someone were to build on your paper and try to improve it, where should they look?
"It's often hard to get students to actually look at outputs and forcing this as part of the game early on is a good idea.”