This Question have 3 answers right now.

Normalise between 0 and 1 ignoring NaN


For a list of numbers ranging from x to y that may contain NaN, how can I normalise between 0 and 1, ignoring the NaN values (they stay as NaN).

Typically I would use MinMaxScaler (ref page) from sklearn.preprocessing, but this cannot handle NaN and recommends imputing the values based on mean or median etc. it doesn't offer the option to ignore all the NaN values.


consider pd.Series s

s = pd.Series(np.random.choice([3, 4, 5, 6, np.nan], 100))

enter image description here

Option 1
Min Max Scaling

new = s.sub(s.min()).div((s.max() - s.min()))

enter image description here

I put these in because I wanted to

Option 2

sigmoid = lambda x: 1 / (1 + np.exp(-x))

new = sigmoid(s.sub(s.mean()))

enter image description here

Option 3
tanh (hyperbolic tangent)

new = np.tanh(s.sub(s.mean())).add(1).div(2)

enter image description here

You could try this - it finds the appropriate index via sys.key_constraints and uses sys.index_columns to identify the column in the index

SELECT AS TableName,
    sc.colid AS Id, AS Name,
    sc.xtype AS TypeId,
    sc.prec AS Precision,
    sc.scale AS Scale,
    sc.iscomputed AS IsComputed,
    sc.isnullable AS IsNullable,
    sc.collation AS Collation,
    (   select count(*) from sys.key_constraints kc
        inner join sys.indexes i on =
        inner join sys.index_columns ic on ic.object_id = kc.parent_object_id and ic.index_id = i.index_id
        where kc.type = 'PK' and kc.parent_object_id = and ic.column_id = sc.colid ) as IsPrimaryKey

FROM syscolumns sc
JOIN sysobjects so ON =
WHERE so.xtype = 'U'
AND = 'Person'

The output is:

TableName   Id  Name    TypeId  Precision   Scale   IsComputed  IsNullable  Collation               IsPrimaryKey
Person      1   Id          56      10          0       0               0    NULL                   1
Person      2   Name        231     50          NULL    0               0    Latin1_General_CI_AS   0
Person      3   Age         56      10          0       0               0    NULL                   0
By : John D

in the constructor, you just need to do this.state = { video: {} } not this.setState

    this.state = {

this.setState can be used anywhere else but the constructor. non es6 way of doing this was doing:

getInitialState: function() {
  return { 
    video: {}

which is equivalent to just doing this.state = {} in the constructor

By : finalfreq

Numpy hstack - “ValueError: all the input arrays must have same number of dimensions” - but they do

MultinomialNB in python/pandas returns “objects are not aligned” error when predicting

Pandas Ordered Histogram Matrix

sckit-learn fit() leads to error after normalising the data

Ordered Logit in Python?

Scikit-Learn's Pipeline: A sparse matrix was passed, but dense data is required

Regression with big data

Cross validation in logistic regression

Gradient Boosting Classifier sparse matrix issue using pandas and scikit

Convert Pandas Dataframe to numpy for sklearn

How to retain column headers of data frame after Pre-processing in scikit-learn

Video about Normalise between 0 and 1 ignoring NaN