This Question have 3 answers right now.

Normalise between 0 and 1 ignoring NaN

Question!

For a list of numbers ranging from x to y that may contain NaN, how can I normalise between 0 and 1, ignoring the NaN values (they stay as NaN).

Typically I would use MinMaxScaler (ref page) from sklearn.preprocessing, but this cannot handle NaN and recommends imputing the values based on mean or median etc. it doesn't offer the option to ignore all the NaN values.



Answers

consider pd.Series s

s = pd.Series(np.random.choice([3, 4, 5, 6, np.nan], 100))
s.hist()

enter image description here


Option 1
Min Max Scaling

new = s.sub(s.min()).div((s.max() - s.min()))
new.hist()

enter image description here


NOT WHAT OP ASKED FOR
I put these in because I wanted to

Option 2
sigmoid

sigmoid = lambda x: 1 / (1 + np.exp(-x))

new = sigmoid(s.sub(s.mean()))
new.hist()

enter image description here


Option 3
tanh (hyperbolic tangent)

new = np.tanh(s.sub(s.mean())).add(1).div(2)
new.hist()

enter image description here



You could try this - it finds the appropriate index via sys.key_constraints and uses sys.index_columns to identify the column in the index

SELECT 
    so.name AS TableName,
    sc.colid AS Id,
    sc.name AS Name,
    sc.xtype AS TypeId,
    sc.prec AS Precision,
    sc.scale AS Scale,
    sc.iscomputed AS IsComputed,
    sc.isnullable AS IsNullable,
    sc.collation AS Collation,
    (   select count(*) from sys.key_constraints kc
        inner join sys.indexes i on i.name = kc.name
        inner join sys.index_columns ic on ic.object_id = kc.parent_object_id and ic.index_id = i.index_id
        where kc.type = 'PK' and kc.parent_object_id = so.id and ic.column_id = sc.colid ) as IsPrimaryKey

FROM syscolumns sc
JOIN sysobjects so ON sc.id = so.id
WHERE so.xtype = 'U'
AND so.name = 'Person'

The output is:

TableName   Id  Name    TypeId  Precision   Scale   IsComputed  IsNullable  Collation               IsPrimaryKey
Person      1   Id          56      10          0       0               0    NULL                   1
Person      2   Name        231     50          NULL    0               0    Latin1_General_CI_AS   0
Person      3   Age         56      10          0       0               0    NULL                   0
By : John D


in the constructor, you just need to do this.state = { video: {} } not this.setState

constructor(){
    super();
    this.state = {
        video:{}
    };
}

this.setState can be used anywhere else but the constructor. non es6 way of doing this was doing:

getInitialState: function() {
  return { 
    video: {}
  }
}

which is equivalent to just doing this.state = {} in the constructor

By : finalfreq



Numpy hstack - “ValueError: all the input arrays must have same number of dimensions” - but they do

MultinomialNB in python/pandas returns “objects are not aligned” error when predicting

Pandas Ordered Histogram Matrix

sckit-learn fit() leads to error after normalising the data

Ordered Logit in Python?

Scikit-Learn's Pipeline: A sparse matrix was passed, but dense data is required

Regression with big data

Cross validation in logistic regression

Gradient Boosting Classifier sparse matrix issue using pandas and scikit

Convert Pandas Dataframe to numpy for sklearn

How to retain column headers of data frame after Pre-processing in scikit-learn

Video about Normalise between 0 and 1 ignoring NaN