Day 23: A Big Day For NLP

New NLP News - Conversational AI tutorial, RNNs for particle physics, InfoGAN, NLP Coursera, NLP book, killer robots, Code2pix, Google AI principles, relational reasoning https://t.co/5zT3QpjYAT

— Sebastian Ruder (@seb_ruder) June 11, 2018

What I've been working on for the past year! https://t.co/CAQMYS1rR7

Inspired by CoVE, ELMo, and ULMFiT we show that a single transformer language model can be finetuned to a wide variety of NLP tasks and performs very well with little tuning/tweaking.

— Alec Radford (@AlecRad) June 11, 2018

This is exactly where we were hoping our ULMFit work would head - really great work from @OpenAI! 😊

If you're doing NLP and haven't tried language model transfer learning yet, then jump in now, because it's a Really Big Deal. https://t.co/0Dj8ChCxvu

— Jeremy Howard (@jeremyphoward) June 11, 2018

Great work! Language models serving as the basis for transfer learning in task agnostic NLP is really going to feed the next generation of tools and transformations =]

— Smerity (@Smerity) June 11, 2018

We show large language models trained on massive text corpora (LM1b, CommonCrawl, Gutenberg) can be used for commonsense reasoning and obtain SOTA on Winograd Schema Challenge. Paper at https://t.co/aRndlByWfj, results reproducible at https://t.co/jFOmUYf03O pic.twitter.com/s3uyrksAQz

— Trieu H. Trinh (@thtrieu_) June 11, 2018

did terminology move and i didn't notice? do you consider language modeling (training a probabilistic model p(w1 w2 w3 ... wN)) to be unsupervised learning?

— Hal Daumé III (@haldaume3) June 12, 2018

Our new paper, Probabilistic FastText for Multi-Sense Word Embeddings, is appearing as an oral at #ACL2018, with code! https://t.co/WoYzYLdd5d
We learn density embeddings that account for sub-word structure and multiple senses. Joint with Ben Athiwaratkun and @AnimaAnandkumar! pic.twitter.com/Xl1MkzJIbt

— Andrew Gordon Wilson (@andrewgwils) June 11, 2018

Graph Convolutional Neural Networks for Web-Scale Recommender Systems. Our new #KDD2018 paper with @PinterestEng. https://t.co/rvFhja3PvO

— Jure Leskovec (@jure) June 11, 2018

"We deploy PinSage at @Pinterest and train it on 7.5 billion examples on a graph with 3 billion nodes representing pins and boards, and 18 billion edges" https://t.co/woN0ZY769t

— Monica Rogati (@mrogati) June 11, 2018

CartoonGAN: Generative Adversarial Networks for Photo Cartoonization (CVPR 2018) https://t.co/xISVUHNbha pic.twitter.com/NfEsbMiQzZ

— hardmaru (@hardmaru) June 12, 2018

Looking for a piece about autonomous weapon systems that manages to link Shakespeare, war dogs, and swarming drones? Looks no further - "Autonomous Weapon Systems and the Limits of Analogy" was published today!https://t.co/zfMqKIypeL

— Rebecca Crootof (@RebeccaCrootof) June 11, 2018

Code, pre-trained models and paper: "QuaterNet: A Quaternion-based Recurrent Model for Human Motion" by Dario Pavllo, David Grangier and Michael Auli.

Code: https://t.co/8pEGUAKNeZ
Paper: https://t.co/YA5etbCJdW https://t.co/J9MZ3NvyTE

— Yann LeCun (@ylecun) June 12, 2018

Excited to share our recent work showing that pooling is neither necessary nor sufficient for appropriate deformation stability in CNNs! Rather, filter smoothness most directly modulates deformation stability in networks both *with* and *without* pooling. https://t.co/nJvvyGenV0 pic.twitter.com/UAmuXbVXbZ

— Ari Morcos (@arimorcos) June 6, 2018

Very interesting paper that uses the Frobenius norm of the difference of the trained weights and their init in a Rademacher complexity bound. Can explain the counter intuitive improved generalization of wider nets. https://t.co/vWWh3M4lit

— Olivier Grisel (@ogrisel) June 11, 2018

Deep Learning Research Review: Natural Language Processing https://t.co/aoBz43ZSMQ pic.twitter.com/bv4gYaBzlh

— KDnuggets (@kdnuggets) June 11, 2018

Slides from my talk earlier using Atari/Go as case studies of recent AI progress. See our recent paper (my pinned tweet) and my blog posts for more on these themes. https://t.co/3nHjq5QtJn

Video here: https://t.co/He9FxnJtYr (orange slides)

Thx @azeem et al. for hosting!

— Miles Brundage (@Miles_Brundage) June 11, 2018

I'm laughing so hard at this slide a friend sent me from one of Geoff Hinton's courses;

"To deal with hyper-planes in a 14-dimensional space, visualize a 3-D space and say 'fourteen' to yourself very loudly. Everyone does it." pic.twitter.com/nTakZArbsD

— Robbie Barrat (@DrBeef_) June 10, 2018

👍: the updated @rawgraphs comes with bee swarm and contour plots https://t.co/HC0jzilvZv pic.twitter.com/diB5dM7rsh

— Maarten Lambrechts (@maartenzam) June 11, 2018

Kernel Trick https://t.co/eZ2bbpDzwV pic.twitter.com/eqAJ8IRGna

— Chris Albon (@chrisalbon) June 11, 2018

Description of 4th place and fastest solution in @Lyft Perception Challenge with #PyTorch code https://t.co/kLsZoV2CUg
- simulated data for semantic segmentation
- hardware limitations
- LinkNet34
- postprocessing on GPU
- speed-up tricks to get >20FPS on single K80 pic.twitter.com/7XaMzu5uZv

— Alexandr Kalinin (@alxndrkalinin) June 11, 2018

Always super helpful: "Basic Regular Expressions in R" by Ian Kopacka https://t.co/WyfezkISPh #RegEx #rstats (#SoDS18 clutch @kierisi) pic.twitter.com/6UY96B8rVu

— Mara Averick (@dataandme) June 11, 2018

Key difference between rbind and dplyr's bind_rows: One refuses to add when columns aren't matching, the other one looks for where the match is (with warnings for missing columns). Depending on the scenario, either one can come in handy #rstats #DSLearning pic.twitter.com/PQyNsT2JeY

— Suzan Baert (@SuzanBaert) June 11, 2018

ICYMI, 👍 code-through:
"Supervised vs. Unsupervised Learning: Exploring Brexit w/ PLS & PCA" by @gokhan_ciflikli https://t.co/iGEZCxFLIu #rstats #caret #tmap pic.twitter.com/jx902RUjBl

— Mara Averick (@dataandme) June 11, 2018

TensorFlow Serving has released version 1.8.0 & now supports a RESTful API out of the box! You can now easily make classification, regression & prediction requests to your TensorFlow models using JSON objects.

Learn more here ↓ https://t.co/BtvgZLi2bb

— TensorFlow (@TensorFlow) June 11, 2018

New #dataset of 5.5 million tweets covers the suicides of Anthony Bourdain and Kate Spade and covers a period of a few days before Kate's suicide to a few days after Bourdain's. Description file ends with .txt and gives details. 30 GB.https://t.co/jaF7rxXGSJ#datascience

— Jason Baumgartner (@jasonbaumgartne) June 11, 2018

Description file located here: https://t.co/vVUZTWYMQY

— Jason Baumgartner (@jasonbaumgartne) June 11, 2018

ICYMI last week we released the largest git dataset in the history of #MLonCode!

Repos contain objects and metadata for all existing revisionshttps://t.co/Aa5Qq8p6bf

— source{d} (@sourcedtech) June 11, 2018

The General Video Game AI (@gvgai) competition lets you compete not only in AI for playing games, but also for making them!
The Level Generation track challenges you to create level generators:https://t.co/DLrWA1yJZy
The Rule Generation track:https://t.co/sAD6vkgMdW pic.twitter.com/PgUqY2G3qB

— Julian Togelius (@togelius) June 11, 2018

Nice talk from @karpathy about the challenges of building the Software 2.0 stack. https://t.co/n4Z4bNhtuG pic.twitter.com/S8nPitGlWt

— hardmaru (@hardmaru) June 11, 2018

my thoughts on the trouble with #d3js https://t.co/1JfLFHl1Lt
to me it's about setting our expectations for learning, and guiding folks on different paths in different ways pic.twitter.com/pZ67Krs0CX

— Ian Johnson (@enjalot) June 12, 2018

I'm happy to report that hyper-intelligent AI isn't that close: currently Amazon, Microsoft, and Google's cutting-edge APIs all believe "My leg was eaten by a great white shark" is a sentence with a highly positive sentiment. pic.twitter.com/rWSvyek9zs

— Jonathan Nolis (@skyetetra) June 11, 2018

Prioritize AI dangers:
- Important and urgent: fight mass manipulation
- Important but not urgent: fight Terminator
- Not important but urgent: stop using stock pictures of robots in every &$!#% article about AI
- Not important, not urgent: stop calling 3-layer neural nets "deep"

— Aurélien Geron (@aureliengeron) June 11, 2018

When the AI bubble bursts by John Langfordhttps://t.co/8qIKLUyo4K

"What fraction of the field is currently in units at companies which are very expensive without yet justifying that expense? It’s no longer a small fraction so there is a chance for something traumatic"

— Giorgio Patrini (@GiorgioPatrini) June 11, 2018

In 2012 my wife and I published a paper using text-mining of neuroscience papers to generate hypotheses.

One prediction was a migraine/striatum link. 3-fold increase in publications since that prediction.

Small-n... need to extend. But neat first-pass!https://t.co/IVNv757trp pic.twitter.com/1norX4OidM

— Brad Voytek (@bradleyvoytek) June 11, 2018

'Luck is statistics taken personally.' -- Penn Jillette

— Data Science Fact (@DataSciFact) June 11, 2018